
.(Jl^J)3 







UNIVERSITY OF PENNSYLVANIA 



STANDARDIZED EXERCISES IN UNITED 
STATES HISTORY. 

COLONIAL PERIOD 



liV 

STURGISS BROWN DAVIS 



A THESIS 

PRESENTED TO THE FACULTY OF THE GRADUATE SCHOOL IN 

PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR 

THE DEGREE OF DOCTOR OF PHILOSOPHY 



UNIVERSITY OF PITTSBURGH PRESS 
1921 



UNIVERSITY OF PENNSYLVANIA 



STANDARDIZED EXERCISES IN UNITED 
STATES HISTORY. 

COLONIAL PERIOD 



RY 
STURGISS BROWN DAVIS 



A THESIS 

PRESENTED TO THE FACULTY OF THE GRADUATE SCHOOL IN 

PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR 

THE DEGREE OF DOCTOR OF PHILOSOPHY 



UNIVERSITY OF PITTSBURGH PRESS 
1921 



Gift 

UrdverElt^ 



Table of Conten is 

1. Introduction 5 

2. Purpose of the Study 6 

Chap. I. Selection and Organization of the Material to be Stand- 
ardized. 

Sec. 1. The Content of the Exercises <) 

Sec. 2. The Form of the Exercises 11 

Chap. II. Standardizing the Material. 

Sec. 1. Procedure Followed 15 

Sec. 2. Schools Taking Test I 20 

Sec. 3. Schools Taking Tests II and III 21 

Sec. 4. Scoring the Exercises 28 

Sec. 5. Tabulating the Results 29 

Chap. III. Fentative Scales in the form of Lists X and Y 33 

Chap. IV. Weighting the Results of the Data on each Exercise. 

Sec. 1. Plan of Procedure 43 

Sec. 2. Tests for the Reliability of the Data 44 

Sec. 3. The Reading Difliculty of the Exercises 50 

Sec. 4. The Element of Success hy Chance. Formula 52 

Chap. V. Classification of the Standardized Material Into Scales C 

and D and Tests E and F 55 

Chap. VI. Use of the Standardized Exercises for Supervisory Purposes. 

Sec. 1. Reorganization of Proposed Measures into Test A 

and Test B, Part I and Part II 69 

Sec. 2. Value of the Standardized Exercises 81 

Sec. 3. Significance of the V'alue Found for Each Exercise. 8 5 
Four Different Kinds of Historical Material 87 

Sec. 4. Possible Uses of the Standardized Exercises 89 

Chap. VII. Summary and Conclusion 95 

Sec. 1 . Summary 95 

Sec. 2. Conclusion 96 

Bibliography 97 



STANDARIZED EXERCISES IN HISTORY 
INTRODUCTION 

There arc now available for the use of teachers a number of 
standardized tests in United States History. All of these tests cover 
in a general way the whole period of history and are especially use- 
ful for survey work. There is need, however, of a different kind 
of test for supervisory purposes which shall include the many details 
of a single period of history and at the same time lend itself readily 
for diagnostic purposes. Such tests have been devised and are here 
presented as one more means of determining scientifically the attain- 
ment of pupils in this subject. 

The plan used in standardizing the material of tests is given 
in more than usual detail in the hope that teachers of history and 
supervisory officers may be able to profit by the experience of the 
writer and extend the work here started. Care has been taken to 
exercise simplicity and directness at all times so that the student 
who is just beginning research work in education may find directly 
or indirectly answers to questions of statistical procedure. 

Briefly the study was carried out as follows: After determining 
tentatively the aims and limits of the investigation, the material to 
be standardized was selected from a source of unquestioned value. 
This material was then put in the form of exercises to be worked 
out by pupils. The responses of these pupils were then scored, tabu- 
lated, evaluated, and finally weighted in terms of the functions of 
the Normal Frequency Curve. These weightings confirmed the 
belief in the possibility of classifying the exercises into tests and scales. 
They also showed that there are four distinct types of historical 
material, characterized by the difficulty pupils have in comprehend- 
ing them. Such classifications were made both in form and content. 
While these classifications are correct theoretically it was deemed 
wise for practical purposes to consider only the approximate weight- 
ing of each exercise and to reclassify all the standardized material 
into two tests. In this form some accuracy was sacrificed but the 
reliability for supervisory purposes is just as valid. A Teacher's 
Manual was prepared which gives full directions for administering, 
scoring and using the results of the two tests formulated for school 
use. 



6 Standardized Exercises in History 

llie execution of this plan would not have been possible with- 
out the hearty co-operation of teachers of history, supervisors, and 
students of education. There was the most cordial response at all 
times and I wish here to express my appreciation of all the assist- 
ance I received however inconspicuous it may have seemed to the 
one giving it. 

Such general acknowledgment, however, is not just regarding 
the S5'mpathetic guidance and suggestions I received from Dr. Harlan 
Updegraff of the University of Pennsylvania. He suggested the 
intensive study of the Colonial Period, and the distinctive source of 
the content of the exercises. He also gave me the benefit of many 
constructive criticisms before the manuscript was presented for 
publication. 

To my fellow students of the seminar group in Educational 
Administration at the University of Pennsylvania, I am indebted in 
more ways than 1 can express in a formal work of this kind. How- 
ever, it should be said that the constructive criticisms and pointed 
suggestions which they offered \vhen the reports of progress were 
made from time to time on this study, served as a valuable check 
against biased thinking and as a stimulus to merit their approbation. 

PL RPOSE OF THE STUDY 

The purpose of this study has been the derivation of stand- 
ardized testing material in United States History comparable in 
exactness with the contents of such material as the Monroe Reading 
Test, the Buckingham Spelling Scales, and the Ayres Handwriting 
Scales. The hypothesis was maintained that it is just as possible 
to measure certain aspects of content subjects, such as history, as it 
has been found to be possible to measure the attainments of pupils 
in the form subjects just mentioned. 

Limits of the Investigation 
For the purposes of this study and under the conditions it had 
to be pursued, it was deemed advisable to circumscribe the field of 
investigation, and to limit the method of work. Only in this way 
did it seem possible to secure reliable results in a field so little known. 
After a careful consideration of the possibilities of the situation the 
following limits were determined : 



In/roduriioTi 7 

1. Informntioiial Material. The content of the proposed tests 
should be confined to the informational aspect of the instruction in 
United States History. Full and due consideration was given to 
the other aims in teaching history, but the attainment of information 
was considered fundamental to the other aims. Dewey has shown 
clearly that information is one of the prerequisites to sound thinking 
on any problem. It seemed evident also that this aim could be 
measured more definitely than the other aims such as citizenship, 
open mindedness, method of thinking, etc. 

2. Otie Period of United States History. The informational 
material used for the tests should be limited to a certain period of 
history, or a cross section of the whole course of study. It was 
thought that the material of a given period would be more homo- 
geneous and could be handled more accurately. The Coloiu'al 
Period was chosen because it has always been taught rather thor- 
oughly In schools, and abounds in informational matter. There 
seems to be a need also for a series of tests each based upon a single 
period of United States History for the use of supervisors after a 
class has been given the usual school instruction. 

3. A Regular Form. The information used for the testing 
material should be cast in a certain form in order that the scoring 
might be objective, and at the same time secure the probability of 
consistency and uniformity in preparing the tests, and the responses 
of pupils. Such uniformity would also be likely to economize time. 

4. Eighth Grade Pupils. The tests should be administered 
only to the advanced division of pupils in the Eighth Grade. The 
purpose in confining the data to this class of pupils was to find out 
as nearly as possible the net results of school instruction relating to 
historical information. The belief was maintained that whatever 
information pupils had, at least one year or more after completing 
the course in the Colonial Period, represented material which is 
easily comprehended and retained ; conversely, that material which 
is difficult to learn would either be wanting or so vague in the minds 
of the pupils as to be lacking for ready command. 



CHAPTER I 

THE SELECTION AND ORGANIZATION OF THE 

MATERIAL TO BE STANDARDIZED 

After the purposes and limits of a study of this kind have been 
determined the next points to be considered are the source of mate- 
rial and the form in which the content is to be placed. Unless this 
is done the investigator must be prepared to expect all sorts of 
irregularities in the responses of pupils. 

Sec. 1. — Contefit of the Exercises. The reliability of the re- 
sponses of children as data for standardizing material of instruction 
is in proportion to the opportunity children have had to learn the 
content of the test. The difficulty of finding such material for 
this investigation was solved by the use of the very careful study 
by Bagley and Rugg. This study appeared as Bulletin No. 16 of 
the School of Education of the University of Illinois. The authors 
took twenty-three American Histories then in common use and made 
a statistical study of the amount of space devoted to the different 
topics, "common to at least seventy-five per cent, of the books" and 
additional topics "common to at least fifty per cent, of the books." 
Tables V and VIII of Bulletin No. 16 include topics devoted to 
the "Period of Colonial Settlement and Development" and to the 
"Colonial Wars." The topics in these tables form the source of the 
content of the exercises used in this study. As the study progressed 
it was evident that additional exercises could be devised which were 
entirely within the range of pupils' interests, and as fair to them 
as the exercises based upon topics taken directly from the Bagley 
and Rugg study. The answers to these exercises were all taught 
in school in some form though seldom if ever asked for directly. 
The ability to work them out was more of an application of common 
sense in using the facts already acquired. The following is a typical 
illustration: "By the time the Mayflower had arrived at Plymouth 
Rock it had crossed the Atlantic, Antarctic, Arctic, Indian, Pacific 
ocean." "By the opening of the Revolutionary War the number of 
colonies had increased to tivelve, thirteen, fourteen, fifteen, sixteen." 
Since these exercises were worked out successfully by a large percent- 
age of the pupils tested, and no objection was raised by teachers, the 



10 Standardized Exercises in History 

author believed he was justified in including them and others of the 
same type. (Copy of Test I is shown beginning on page 18.) To 
aid the reader in gaining a more adequate conception of the wide 
range of information required to work out intelligently the exer- 
cises of Test 1 the following classification of the items of informa- 
tion is here submitted : 



NOTED CHARACTERS 



1. 


Andros 


15. 


Davenport 


29. 


Mather 


2. 


Argall 


16. 


Delaware 


30. 


Oglethorpe 


3. 


Baltimore 


17. 


Dixon 


31. 


Penn 


4. 


Berkley 


18. 


Edwards 


32. 


Pocahontas 


5. 


Berkeley 


19. 


Endicott 


33. 


Powhatan 


6. 


Bacon 


20. 


Gorges 


34. 


Rolfe 


7. 


Bradford 


21. 


Hooker 


35. 


Say 


8. 


Brewster 


22. 


Hutchinson 


36. 


Smith 


9. 


Calvert 


23. 


Kieft 


37. 


Standish 


10. 


Canonicus 


24. 


Leisler 


38. 


Stuyvesant 


11. 


Carteret 


25. 


Locke 


39. 


Williams 


12. 


Charles 


26. 


Mason 


40. 


Winthrop 


13. 


Claiborne 


27. 


Massasoit 


41. 


Yeardley 


14. 


Dale 


28. 


Minuit 










RELIGIOUS 


SECTS 




1. 


Baptists 






7. Moravians 


2. 


Catholics 






8. Pilgrims 




3. 


Episcopal 






9. Puritans 




4. 


Dutch Reformed 






10. Quakers 




5. 


Huguenots 






11. Congregationalists 


6. 


Methodists 
















COLLEGES 




1. 


Brown 






4. Princeton 




2. 


Harvard 






5. William 


and Mary 


3. 


Penn 






6. Yale 








EARLY LARGE CITIES 




1. 


Baltimore 






4. Philadelphia 


2. 


Boston 






5. New York 


3. 


Charleston 
















NATIONALITIES 




1. 


Dutch 






5. Irish 




2. 


English 






6. Scotch 




3. 


Germans 






7. Spanish 




4. 


Indians 






8. Swedes 







Selection 


and Organization 








DATES 


1. 1607 






6. 1733 


2. 1619 






7. 17S4 


3. 1620 






8. 1775 


4. 1643 






9. 1787 


5. 1683 






10. 1789 






LEGAL TOPICS 


1. Bacon's Rebellion 






11. Aristocracy 


2. Charter Oak 






12. Autocracy 


3. Charter Colonies 






13. Oligarchy 


4. Claiborne's Rebell 


ion 




14. English Crown 


Colonial JVars 






15. Continental Congress 


5. Anne 






16. Grand Model 


6. William 






17. Great Treaty 


7. George 






18. Mason and Dixon Line 


8. French and Ind 


ian 




19. Royal Government 


9. Democracy 






20. Toleration Act 


iO. Monarchy 












EARLY COLONIES 


1. Connecticut 






8. New Jersey 


2. Delaware 






9. New York 


3. Georgia 






10. Pennsylvania 


4. Maryland 






11. Rhode Island 


5. Massachusetts 






12. South Carolina 


6. New Hampshire 






13. Virginia 


7. North Carolina 






14. Maine 



11 



1. Boston News Letter 

2. City of Brotherly Love 

3. Debtors of England 

4. First Colony 

5. Indigo 

6. Indentured Servants 

7. Holy Experiment 

8. Last Colony 

9. Log House 

10. Mayflower 

11. New England 



MISCELLANEOUS 

12. Patroons 



13. Plymouth Rock 

14. Poor Richard's Almanac 

15. Preachers' Influence 

16. Rice 

17. Slaves 

18. Tobacco 

19. Witchcraft 

20. Jamestown 

21. Corn 
Total 136 



Sec. 2. — Form of the Exercises. After the content of the test- 
ing material had been decided upon it was necessary to consider the 
form in which it should be put. The responses of pupils had to be 
evaluated in an objective manner. 



12 Standardized Exercises in History 

The studies of Starch, Monroe, Kelly and others all show how 
variable are the marks of teachers if there is opportunity for a differ- 
ence of opinion. The ordinary "question and answer" method of 
testing pupils was at once discarded for the usual difficulties would 
be inevitable. Either doubtful or partial answers would have to 
be accepted or the content be limited to those few points about which, 
teachers and pupils readily agree there can be but one acceptable 
answer, such as dates. 

The "completion blank" form was considered and tried in a 
preliminary test. It was not found satisfactory, for either the pos- 
sible acceptable answers had to be listed arbitrarily or a shifting 
standard constantly defended. It was noted also that pupils made 
wild guesses, and that it would be very difficult to frame up state- 
ments which did not admit of two or more interpretations. One 
has but to try this plan to see how ingenious pupils are in inventing 
answers which represent a partial truth. At least they contain enough 
excellence for pupils to stand up and defend their answers. 

A cancellation test suggested from those used '\n psychological 
work was tried, but rejected because of the unnecessary length of 
time required to indicate the correct answer. 

Finally it was decided to put the desired response in each case 
along with seemingly correct responses and ask the pupils to indicate 
in some way their choice. At first they were asked to underscore 
the correct answer, but later it was found that it facilitated the 
scoring to have them draw a line clear around the correct one of 
the several suggested answers. A trial with this form proved satis- 
factory from the first. Pupils readily saw what was wanted, teach- 
ers approved it, and the grading was easily and objectively done. 

As the work of forming the material into tests progressed it 
became evident that standards could be set up which if observed 
would greatly facilitate subsequent efforts in this line. The follow- 
ing were derived : 

1. Instead of asking questions, make statements called "exercises" 
in History. 

2. The form and content of these exercises should be unmistakably 
clear and as easy to comprehend as possible. 



Select io?i and Organization 13 

3. The vocabulary employed in the exercises should be such as is 
used in standard text books for elementary schools. 

4. Sufficient suggested answers should be given to minimize the 
effect of chance selection. 

5. An alphabetical arrangement of the suggested answers should 
be observed to avoid any constant error in placing of the correct 
answer. 

6. Pupils should indicate the correct answer by drawing a line 
clear around it. 

7. Any abbreviations used should be those in common use. 

8. The answers offered should conform to the period under con- 
sideration. 

9. The suggested answers should be made up of teaching mate- 
rial, and any one of them seem a plausible answer to one not 
informed on the subject. Absurd suggested answers defeat the 
plan. 

10. The suggested answers should be so different from the correct 
one that any quibbling over partial truths is precluded. 

11. It should be clearly understood that one answer and only one 
will be accepted as correct. 

12. No exercise should be submitted unless there is reasonable evi- 
dence that pupils have had the opportunity to learn the facts 
needed. 

The last five of these standards refer more specifically to con- 
tent but were here included for convenience since both form and 
content must be considered together in making up testing material. 
The term, "Exercises in History," was adopted because it seemed 
to be the most fitting one to vise. The material was not put in the 
usual statement form and it was not in the form of questions. Pupils 
were accustomed to the word exercise as used in Arithmetic, Geog- 
raphy and Language. In each of these instances there was an answer 
to find. The situation is similar in the Exercises in History. A num- 
ber of answers are given and the pupil is expected to find the correct 
one. 



CHAPTER II 
STANDARDIZING THE MATERIAL 

Sec. 1. — Procedure Followed. Having determined the content 
and the form of the testing material it was next necessary to deter- 
mine the specific and relative difficulty of each one of the exercises. 
Two plans have been used by research students in standardizing 
material. One plan is to obtain the combined judgment of teach- 
ers whose experience and training seemingly qualifies them to render 
expert opinion as to the value of given questions, or the quality of 
work done by pupils. The Harvard-Newton Composition Scale and 
the Thorndike Handwriting Scale are outstanding examples of this 
plan as to quality of work. The other plan is to submit the mate- 
rial to pupils and grade the work according to standards acceptable 
to all. The assumption is made that, that material is easy which 
is easily, readily and correctly worked out ; conversely, that testing 
material is difficult which is not easily, readily and correctly worked 
out by the pupils. 

The second one of these plans was adopted for standardizing 
the historical material used in this study. Each of the exercises was 
then submitted to pupils of the eighth grade in well recognized public 
schools. Since the number of responses which were supposedly 
needed, was greater than the writer could procure from pupils 
directly, a form letter and directions for administering the test were 
prepared and sent out to the chief school official of the system from 
which data were desired. A copy of Test I was always included 
with the letter so the administrative officer could see just the nature 
of the data expected. There was a hearty response for co-operation 
in every instance. In so far as it was possible the writer adminis- 
tered the tests himself. 

The form letter and directions for "Classroom Procedure" are 
as follows: 

EXERCISES IN UNITED STATES HISTORY 
Colonial Period 
To Teachers of U. S. History: 

You are in a position to co-operate in an experimental study of 
the historical matter now generally taught in our Elementary 

15 



16 Standardized Exercises in History 

Schools. The Exercises herewith enclosed is the result of a prelimi- 
nary study which, it is hoped, will result in definite suggestions for 
class room work. 

Will you kindly assist in securing actual results from the pupils 
in your charge? 

The appended Classroom Procedure for administering the 
Exercises, and the Directions to Pupils on the first page of the fold- 
ers, are intended to make clear all necessary points. Please read 
these over carefully, and follow them as given. 

The material readily lends itself to the game idea and it is sug- 
gested that it be so used. A correctly marked copy is enclosed for 
your own use. Please do not mark any of the pupils' work, however. 

In working up the results of this study each school will be 
designated by number only. No personalities whatsoever will enter 
into the summary. The median score for all schools listed, and the 
median for your school will be sent you. 

Your assistance will be greatly appreciated by those responsible 
for the results of this study. Return postage is enclosed to cover 
cost of all material sent out. 

Address School of Education, 

University of Pennsylvania, 
Care of Dr. Harlan UpdegrafiF, 

Box 11, College Hall. 

CLASSROOM PROCEDURE 

1. All pupils who are going to participate should clear their desks, 
and provide themselves with pencils. 

2. READ TO PUPILS: "Today we are going to have a n^w 
way of finding out how many facts of the Colonial History of 
our country we know really well. In order that all may have 
the same opportunity of making a good score, no one should 
start the game until I give the signal. I shall now pass out 
the papers with the first page upwards. You may look at fchis 
but nothing more until I say, read y." 

3. Distribute the Exercises. 

4. Direct the pupils to fill out the blanks at the top on the first 
page. 

5. Read over with the pupils the illustrative exercises. If neces- 



Standardizing, the Material 17 

san' put one or more of them on the black-board and work them 
out so as to make sure pupils know exactly just what is to be 
done. 

6. When all is in readiness for a period of uninterrupted work, 
say, Ready, GO! 

7. Note down the time when the class begins working, 

8. As the pupils hand in their papers put down the number of 
minutes each worked on the blank space for that purpose. 

Test I 

UNIVERSITY OF PENNSYLVANIA 

School of Education 

EXERCISES IN UNITED STATES HISTORY 

Colonial Period 

Directions to Pupils: Fill in these blank lines with: 

You r name School 



Grade City Date 

Are you a boy or girl ? 

1. On the following pages are a number of exercises similar to these. 

(a) Columbus discovered Africa, America, Asia, Australia. 

(b) The American Revolution began in the year 1763, 1765, 1775, 

1776. 

(c) The first American colony founded by the English was Conn., 

Ga., S. C, Fa. 

2. If you draw a line around the correct one of the several suggested an- 

swers in italicized letters for each Exercise, historical facts may be 
read as follows: 

(a) Columbus discovered America. 

(b) The American Revolution began in the year 1775. 

(c) The first American colony founded by the English was Va. 

3. There is but one correct answer in each Exercise. 

4. Try to find the correct answer in each Exercise whether you are sure 

of it or not. 

5. Hand in your paper as soon as you have worked out all the exercises 

you can. Take as much time as you need, but no more. 

6. Number of minutes you worked (Your teacher will note this.) 



18 Standardized Exercises in History 

HISTORY OF THE UNITED STATES 

Colonial Period 

Test I 

Exercises 

1. The Mayflower was a chapel, hall, hotel, plant, queen, ship. 

2. Miles Standish led in fighting the Dutch, Indians, Puritans, Swedes. 

3. Roger Williams founded the colony of Ga., Md., Mass., N. C, N. 7,, 

Pa., R. I., S. C, Fa. 

4. The Patroons were Dutch fishermen, fur traders, lando'wners, miners, 

preachers, teachers. 

5. Thomas Hooker led emigrants from Mass. to found Conn., Del., Ga., 

Md., N. J., N. Y. 

6. The first college founded was Broiun, Harvard, Pennsylvania, Princeton, 

JVilliam and Mary, Yale. 

7. The Witchcraft delusion occurred among the Baptists, Catholics. Dutch, 

Indians, Puritans. 

8. John Berkley and George Cartaret once owned Conn., Del., Ga., Md., 

Mass., N. H., N. J., Pa. 

9. The principal native food crop was barley, corn, oats, potatoes, rice, rye, 

ivheat. 

10. The Mason and Dixon Line was established between Pa. and Del., 

Md., N. J., N. Y., Ohio. 

11. Nathaniel Bacon led a Rebellion against the tyranny of Governor 

Argall, Andros, Berkley, Dale, Minuit, JVinthrop, Yeardley. 

12. The Charter Oak was a church, meeting house, painting, school, town 

hall, tree. 

13. The Grand Model was drawn up by Cecil Calvert, King Charles, John 

Locke, John Smith. 

14. Of the nationalities represented in all the colonies the most numerous 

were the Dutch, English, French, Germans, Irish, Scotch, Swedes. 

15. The Proprietary form of government was exercised in Mass., N. Y., 

Pa., R. I., S. C, Fa. 

16. The most influential professional class in New England were the editors, 

lawyers, physicians, preachers, teachers, ivriters. 

17. The Toleration Act in Md. was decreed by the Episcopalians, Catholics, 

Purtians, Quakers. 

18. To help the poor debtors of England James Oglethorpe founded the 

colony of Conn., Del., Ga., Md., N. J., S. C, Fa. 

19. The Pilgrims were kindly received by Chief Canonicus, Massasoit, 

Philip, Powhatan. 

20. The first settlement by the Swedes was in Conn., Del., Ga., Md., Mass., 

N. C, N. H., N. Y. 



Stmulardizing the Material 19 

21. Slavery was first introduced in J607, 1619, 1620, 1643. 1683, 1775, 1787. 

22. New York was founded by the Dutch, English, French, Spanish, Swedes. 

23. The main export crop of early Va. was corn, cotton, indigo, rice, tobacco, 

<wheat. 

24. The Great Treatj- was between the Indians and Governor Berkley, John 

Endicott, Cotton Mather, IFilliam Penn, John Smith, Miles Standish. 

25. Pocahontas is said to have saved the life of Sir IVilliain Berkley, Jona- 

than Edivards, JVilliam Penn, John Rolfe, John Smith. 

26. The Charter form of government was enjoyed by the people of Del., 

Ga., Md., Pa., R. I., Va. 

27. The Huguenots first settled in Baltimore, Boston, Charleston, Neiv York, 

Philadelphia. 

28. The "Plymouth Rock" is in Conn., Del, Md., Mass., N. J., N. Y., Pa., 

R. I., S. C. 

29. The first Pilgrims landed in 1607. 1619, 1620, 16i3, 1683, 1776, 1789. 

30. William Claiborne led a Rebellion against Lord Berkley, Lord Brook, 

Lord Baltimore, Lord Delaware, Lord Say. 

31. Indentured servants and slaves were held mostlv in Del., Ga.. Mass., 

Pa., S. C, Fa. 

32. As Royal Governor of New En-^land Sir Edmond Andrns represented 

the Continental Congress, the English Crown, the Proprietors, the 
Common People. 

33. The First Representative Assembly was held in America in 1607, 1619 

1620, 1643. 1754. 

34. William Bradford was once Leader of the Baptists, Catholics, Pilgrims, 

Quakers. 

35. The first settlement of the Catholics was in Conn., Del., Md., Mass., 

N. C, N. H., N. J., R. L 

36. Jamestown was founded in 1607, 1619, 1620. 1643. 1733. 1754. 

37. The first Newspaper in America was the Boston News Letter, New York 

Times, Philadelphia Ledger, Poor Richard's Almanac, Providence 
Journal. 

38. The houses of the first colonists were generally built of brick, cement. 

dirt, marble, logs. 

39. The last of the thirteen colonies founded was Conn., Del., Ga., Md., 

Mass.. N. C, N. Y., Pa., S. C, R. /., Va. 

40. In emigrating the colonists generally went east, north, south, west. 

41. The last of the Dutch governors was Hudson, Minuit, Stuyvesant, fVin- 

throp, Yeardley. 

42. The Germans settled mostly in Md., N. J., N. Y., Pa., R. I., S. C, Va. 



20 Stdndardizcd Exercises in History 

43. The City of Brotherly Love was founded by IVilliam Berkley, George 

Calvert, John Endicott, If'illiam Perm. John Smith, Miles Standish. 

44. The most important of the Four Inter-Colonial Wars to the colonists 

was Queen Anne's, King George's, French and Indian, King 
JFilliam's. 

45. By the middle of the 18th century the form of government most desired 

by the colonists was aristocracy, autocracy, democracy, monarchy, 
oligarchy. 

46. The Quakers in Mass. were persecuted by the Baptists, Methodists, 

Moravians, Puritans, Dutch Reformed. 

47. The Colony of Ga. was once invaded by the Dutch, French, Irish, 

Spanish, Svcedes. 

48. The colony of Me. was once under the proprietorship of John Davenport, 

Sir Ferdinando Gortjes, Inne Hutchinson, U'iUiam Kieft, John 
Davenport. 

49. The "Holy Experiment" was tried out by irilliam Bradford, William 

lircii^ster, Jacob Leisler, Peter Minuit, irilliam Penn. 

50. When the colonists first landed in America they found the nrstives to be 

Caucasians. Chinese, Indians, Malays, Negroes. 

SfiC. 2.— Schools Taking Test I. In the spring of 1918 the 
writer sent out or administered personally approximately 1,500 copies 
of Test I to schools in Massachusetts, New Jersey. Pennsylvania, 
Ohio, Indiana, Illinois, and Minnesota. Those administered by the 
writer were in schools in or near Philadelphia. The returns from a 
distance came from school officials who were interested in educational 
progress and there is every reason to believe that the test was admin- . 
istered according to directions. In all 1,250 papers were returned, 
corrected and tabulated by the writer. The others were lost in one 
way or another. 

In the spring of 1919 the writer administered personally 
approximately 1,500 more of the same test in schools in and near 
Pittsburgh. This district is distinctly cosmopolitan, and has the 
further advantage for experimentation of variation resulting from 
the complex school organization of former years. Allegheny County 
is so populous and wealthy that many independent school districts 
are supported. The "State of Allegheny" has been aptly applied to 
this region. Even in the city itself, Pittsburgh, independent char- 
acteristics of instruction may yet be noted among those holding 
over from the ward system, although that form of organization was 
years ago supplanted by a modern unified system. The results from 



Standardizing the Material 21 

this region within a few miles distance show as wide a variabih'ty as 
those gained from different states. 

When the data from this comparatixely limited area had h'cw 
scored, tabulated and weighted, they were found to be as valuable 
as the data coming the previous year from the seven states extendiuR 
as far east as Massachusetts and as far west as Minnesota. This 
new material was all the more striking since it was procured in 
different years. 

Table I shows how closely the results for the two different years 
and from such widely separated school systems correspond. Using 

the Pearson cos. method of correlation in which r:=cos ^ ' .? 

be 

the coefficient is found to be 1.00. If Sheppard's method of unlike 

signs be applied in which r=cos : , the coefHcient is foun.d to 

^ L _L IT ' 

be 1.00. (^t course these formida? are for approximate results, but 
they are convenient for giving a quantitative statement about th? 
data in this table which by inspection shows remarkable agreement. 
Ry Spearman's "Foot Rule," r=.98. 

The data producing these results came from pupils who were 
ready for high school in the following June. 

The values in Table I for 1918 are based upon the errors made 
by 1,250 pupils; the values for 1919 are based upon the errors made 
by 1,250 additional pupils, 2,500 in all. 

Table II is a modification of Table I and brings out other inter- 
esting details explained in the table headings 

Fig. I shows in a graphic form the distribution of errors made 
by the 2,500 pupils. The close approximation to the Normal Curve 
of Error may be noted. 

Taking the test as a whole the addition of the data in the 
second year made but few striking changes. In many cases the rank- 
ing is identical. The algebraic svim of the shifting differences is 
zero. The author had additional data from pupils to add to that 
gathered in 1919, but when the results turned out so little different 
from those of the previous year it was not considered necessary to 
use them. 

Sec. 3. — Schools Taking Test II and III. Although the re- 



22 



Standardized Exercises in History 



TABLE I 

COMPARISON OF THE SIGMA VALUES FOUND FOR EACH OF THE 

EXERCISES OF TEST I 



No. of 


Value 


Value 


No. of 


Value 


Value 


Exercise 


1918 


1919 


E.cercise 


1918 


1919 


1 


.53 


.43 


26 


3.00 


3.03 


2 


1.94 


2.05 


17 


2.84 


2.77 


3 


1.96 


2.16 


28 


1.90 


1.95 


4 


2.43 


2.44 


29 


2.25 


2,37 


^ 


2.31 


2.39 


30 


3.15 


2.7t 


6 


2.40 


2.37 


31 


2.36 


2.31 


7 


1.98 


2.12 


32 


2.42 


2.38 


S 


3.39 


3.32 


33 


3 . 43 


3.36 


9 


1.75 


1.85 


34 


2.76 


2. SO 


10 


2.48 


2.46 


35 


2.28 


2.34 


11 


2.39 


2.39 


36 


1.85 


2.05 


12 


1.50 


1.46 


37 


3.49 


3 i2 


\} 


3.49 


3.35 


38 


.7^' 


.73 


14 


1.57 


1.64 


39 


2.62 


2 66 


15 


3.03 


3.01 


40 


2.05 


1 94 


16 


2.65 


2.61 


41 


2.13 


2.04 


17 


2.86 


2.77 


42 


2.72 


2.51 


IS 


2.25 


2.29 


43 


1.46 


1.41 


L' 


1.99 


2.04 


44 


1 . 59 


l.Sx 


20 


2.55 


2.55 


45 


1.50 


1.40 


21 


2.03 


2.26 . 


46 


2.23 


2.28 


22 


1.27 


1.35 


47 




2.42 


23 


1.73 


1.86 


48 




3 . 26 


24 


1.43 


1 41 


49 




2.24 


25 


1.19 


1.24 


50 




.51 



suits from Test I were satisfactory the exercises were too uniform 
in value to construct a series of standardized tests or scales. By 
reference to Table II it can be seen that the nearest approach to the 
zero value is .43 sigma ; and the nearest approach to the other ex- 
treme of the scale, 5.0, is 3.36 sigma. 

In the fall of 1919 Test II was devised with the purpose of 
finding, if possible, well recognized material whose value would be 
more varied than that yet discovered. Some exercises were then 
devised which were thought to be very easy for pupils to work out, 
and others which were supposedly very difficult. The study of 
Bagley and Rugg was the source of the material as before, and it 
w as sanctioned by teachers as being fair for the pupils. The same 
form was used as previously except that the number of suggested 
answers was kept uniform. Eleven different schools were chosen for 



Standardizing the Material 



23 



TABLE II 

COMPARISON OF VALUES FOR THE EXERCISES OF TEST I 

FOR TWO YEARS 



Number 


Sigma 


Sipma 


Rank 


Rank 


Difference in 


of Each 


Value 


Value 


in 


in 


Ranking for 


Exercise 


in 1919 


in 1918 


1919 


1918 


the two years 












D+ D- 


1 


.43 


.52 


1 


1 





i)i 


.73 


.75 


2 


2 





1> 


1 24 


1.19 


3 


3 





11 


1.35 


1.27 


4 


4 





45 


1.40 


1.50 


5 


7 


2 


24 


I 41 


1.43 


6 


5 


1 


43 


1.41 


1.46 


7 


6 


1 


12 


1 46 


1.50 


8 


8 





44 


1.58 


1.59 


9 


10 


1 


14 


1.64 


1.57 


10 


9 


1 


9 


1.85 


1.75 


11 


11 





23 


1.86 


1.78 


12 


12 





40 


1.94 


2.05 


13 


20 


7 


28 


1 95 


1.90 


14 


14 





41 


2.04 


2.13 


15 


21 


6 


19 


2.04 


1.99 


16 


18 


2 


2 


2.05 


1.94 


17 


15 


2 


36 


2.05 


1.85 


18 


13 


5 


7 


2 12 


1.98 


19 


17 


2 


3 


2.16 


1.96 


20 


16 


4 


21 


2.26 


2.03 


21 


19 


2 


46 


2.28 


2.23 


22 


22 





IS 


2.29 


2.25 


23 


24 


1 


31 


2.31 


2.36 


24 


27 


3 


35 


2.34 


2.28 


25 


25 





.29 


2.37 


2.25 


26 


23 


3 


6 


2.37 


2.40 


27 


29 


2 


5 


2.38 


2.31 


28 


26 


2 


32 


2.38 


2.42 


29 


30 


1 


11 


2.39 


2.39 


30 


28 


2 


4 


2.44 


2.43 


31 


31 





10 


2.46 


2.48 


32 


32 





42 


2.51 


2.72 


33 


36 


3 


20 


2.55 


2.55 


34 


33 


1 


16 


2.61 


2.65 


35 


35 





39 


2.66 


2.62 


36 


34 


2 


30 


2.74 


3.15 


37 


42 


5 


17 


2.77 


2.86 


38 


39 


1 


27 


2.77 


2.84 


39 


38 


1 


34 


2 80 


2.76 


40 


37 


3 


15 


3.01 


3.03 


41 


41 





26 


3.03 


3.00 


42 


40 


2 


S 


3.32 


3.39 


43 


43 





37 


3.32 


3.49 


44 


46 


2 


13 


3.35 


3.49 


45 


45 





33 


3.36 


3.43 1 


46 


44 
Total 


2 
36 36 



24 



Stdudardizeil Exercises in History 




PERCENT OF ALL PUPILS 



Standardizing the Material 25 

the experiment which were in every way comparable with the schools 
in which Test I was given. The same directions were followed as 
before, and the work was all done by the author. Especial care was 
taken in administering, scoring and tabulating the results in order to 
make the outcome as accurate as possible. In spite of all the possible 
precautions the net results of this work revealed no exercises to be 
of lower value or decidedly higher value than were found in Test I. 
It was therefore necessary to devise another test which was des- 
ignated as Test III. The same methods of working were used as 
before except that the pupils were permitted to grade their own 
papers first, and all the material needed for weighting the exercises 
was gathered in the school room according to a plan to be given in 
detail in Chapter III. The results were more satisfactory. Al! 
the exercises in Tests II and III were then carefully evaluated and 
considered as preliminary values to forming two tentative scales. 
In all 1,102 pupils had been tested in 18 different schools. At this 
point are given copies of Tests II and III. The same directions 
were followed in each test. 

Test II 

EXERCISES IN COLONIAL HISTORY 

Draw a line around the one italicized word which is needed to turn 
each exercise into a historical fact. 

1. The "Mayflower Compact" was drawn up by the Baptists, 1 

Huguenots, Methodists, Pilgrims, Quakers. 

2. When the earliest colonists arrived in America they were wel- 2 

corned by the Caucasians, Chinese, Indians, Malays, Negroes. 

3. George Washington was fitted for leadership in the Revolutionary 3 

War through his experiences in the Colonial War of Queen 
Anne, King George, French and Indian, King IFilliam. 

4. By the opening of the Revolutionary War the number of colonies 4 

had increased to tivelve, thirteen, fourteen, fifteen, sixteen. 

5. The first attempt of the English to colonize in America was in what 5 

was later called the colony of Conn., Del., Mass., Penna., Va. 

6. The territorial claims of the English in America were established 6 

by the treaty of Aix-la-Chapelle, American, Paris, Rysvjick, 
Utrecht. 
1. The period of colonization took place largely in the period begin- 7 
ning with 14-00, 1500, 1600, 1700, 1800. 



26 Standardized Exercises in History 

8. The political control of the Dutch in America came to an end with 8 

the surrender of Governor Stujvesant to the English, French, 
Germans, Spaniards, Sivedes. 

9. "Braddock's Defeat" occurred in the first, second, third, fourth, 9 

Colonial War. 

10. The first written constitution springing from the people and creat- 10 

ing a government was in Conn., Del., Mass., Penna., Fa. 

11. The French army built menacing forts in Western Conn., Mass., 11 

Md., N. J., Penna. 

12. The leadership of John Winthrop was noted in the founding of 12 

Baltimore, Boston, Charleston, Philadelphia, Providence. 

13. The most outspoken friend of the colonists in the English Parlia- 13 

ment was Din<ividdie, Montcalm, Pitt, Toivnshend, JVolfe. 

14. Lord Delaware lived in England nearly all the time he was gover- 11 

nor of Delaivare, Massachusetts, Neiv Jersey, Rhode Island, 
Virginia. 

15. The "Pilgrim Fathers" came from England, France, Germany, 15 

Spain, Siveden. 

EXERCISES IN UNITED STATES HISTORY 
Test III 
Colonial Period 
Draw a line around the one italicized word which is needed to turn 
each exercise into a historical fact. 

1. The Pilgrim Fathers came from England, France, Germany, Spain, 1 

Sijjedcn. 

2. The last colony was founded in the 15th, 16th, 17th, 18th, 2 

19 til, century. 

3. The New Sweden of 1638 later became known as the colony of 3 

Conn., Del., N. H., Pa., R. I. 

4. Among the many hardships of the time were the unexpected attacks 4 

by Arabian, Chinese, Cossack, Indian, Negro warriors. 

5. More lasting and prominent political events took place in Baltimore, 5 

Boston, Charleston, Neiv York, Philadelphia than in any other 
city. 

6. The colonists were assisted in the French and Indian Wars by the 6 

Iroquois, Molicgan, Narragansett, Pequot, IVampanoag Indian 
tribe. 

7. The right of woman suffrage was granted for a time in Conn., 7 

Del, Mass., N. J., Va. 

8. Local business affairs in the New England settlements were gen- % 



Standardizing the Material 27 

erally managed bj the governor, king, mayor, minister, toivn- 
meeting. 
9. The welfare of the New York colon}- was much helped by the very 9 
early settlement of the Huguenots at Albany, Ne^ Burg, 
Neiv Roclielle, Saratoga, West Point. 

10. The strict Puritan church of early New England gradually gave way 10 

to what was later known as Baptist, Congregationalist, Lutheran, 
Methodist, Spiritualist Church. 

11. The light at night in these early homes came from candles, elec- 11 

triciiy, gas, kerosene, torches. 

12. The Quakers founded Ga., Mass., N. H., Pa., S. C. 12 

13. By the time the Mayflower arrived at Plymouth Rock, it had crossed 13 

the Antarctic, Arctic, Atlantic, Indian, Pacific ocean. 

14. The most common means of travel by the earliest settlers was bv 14 

aeroplane, railroad, steamboat, trolley lines, tjijalking. 

15. The I'nited Colonies of New England was formed in 1619, 1630, 15 

1643, 1683, 1763. 

16. All the colonies were founded by men from Africa, Asia, AuS' 16 

tralia, Europe, South America. 

17. The "Thirteen Original Colonies" were once all under the political 17 

control of England, France, Holland, Spain, Sivedrn. 

18. Pocahontas was an English, French, German, Indian, Spanish girl. 18 

19. Lord Delaware lived in England nearly all the time he was gov- 19 

ernor of Del., Mass., N. J.. R. I., Va. 

20. The first written constitution springing from the people and creating 20 

a government was in Conn., Del., Mass., N. J., la. 
When you have finished turn your paper over. 

Number of errors 

It will be noted that there is some overlapping in the two tests. 
This was done to see if certain exercises would retain their rank 
when conditions were kept as uniform as possible ; and further to 
see if a few hundred pupils are all that is necessary to ascertain the 
relative difficulty, or the ease of a given point in history. That the 
exercise, "The first written constitution springing from the people 
and creating a government in America was in Conn., Del.. Mass., 
Penna.. Va.," had a value of 3.55 in the first 11 schools and a value 
of 3.49 in the succeeding 7 schools was not due to mere chance. 
The experience with this exercise and the others used in both tests 
showed that if the material is carefully chosen, administered scien- 



28 Standardized Exercises in History 

tifically, and a proportional sampling carefully guarded, only a few 
hundred pupils are necessary to tell the relative difficulty of a list of 
historical points. 

Attention is called to the dual numbering of the exercises in 
Tests II and III and Lists X and Y. The numbering at the right 
was used by the writer in checking over the corrections made by 
pupils. The point to this device is that the writer found it more 
convenient to indicate his marks on the right hand side of the page 
Vv'here the right hand never obstructs the view in a quick survey 
of the page. Hiis last suggestion may seem like a trivial point but 
the observance of such details facilitates the work when the scoring 
runs up into the thousands of exercises to correct. 

Sec. 4. — Sioritu/ the Exercises. The scoring was done by the 
writer or by competent assistants under his direction. In all there 
were about 125,000 responses of pupils to check over respecting Test 
I alone; 6,000 for Test II; 14,000 for Test III; and 46,000 for 
Lists X and Y. Counting the control checks that were used respect- 
ing high school and normal school students more than 192,000 pupils' 
responses were taken into consideration in the scoring to produce the 
results desired. In the case of Tests I and II each exercise was 
looked over to see if it had been worked correctly or not. If an error 
had been made a straight line was drawn obliquely downward at 
the left of the number of the exercise in the case of Test I ; but to 
the right in the case of Test II since the numbers of the exercises 
were written on both sides. This kind of mark was used because 
it is quickly drawn and easily discerned, being away from the hori- 
zontal of the printed lines. 

Only errors were counted. They were less in number taken 
on the whole ; only errors were considered in weighting ; and the 
errors are the points of consideration in diagnosis. The number of 
errors on each paper was totaled and the simi placed at the lower 
right hand corner of the front page. 

As the work of scoring so many thousands of responses soon 
borders on the line of drudgery, several methods were used to expe- 
dite the work. Stencils were used such as are now used in scoring 
some of the current Intelligence Tests; a certain rhythm was main- 
tained by the use of a phonograph ; but the most effectual plan 



Standardizing the Alafrrial 29 

devised was to commit to the point of mechanical memory the correct 
answers as they should appear in order in running down the exami- 
nation paper. Any departure from the regular order or correct 
responses was thus readily and immediately detected. In Lists X 
and Y where pupils corrected their own papers it was necessary only 
to take samplings at random and unusual scores to see if the correc- 
tions needed to be checked over. In Chapter III is given the detailed 
directions which were used when pupils were permitted to mark 
and check over their own work. 

Snc. 5. — Tabulating the Results. After the papers had been 
scored it was necessary to tabulate the results so they could be 
analyzed and interpreted. In this part of the statistical procedure 
the writer aimed to secure accuracy, speed, simplicity and the possi- 
bility of checks at as many points as possible. The plan adopted 
for this study is illustrated in Fig. 2. The tabulations are the actual 
results taken from the errors made by 22 pupils to whom Test II 
was given. 

PUPILS' PAPERS BY NUMBER 





/ 


2 J J 


J- 


tit')'" 


Jt ^t. /i )<)■ /S 


/« n /t '? M) 


2/ AJ~ 






/ 

J 

H 

r 


1 i ! 
iii 

Mi 

1 ! 1/ 






i / i 1 

1/ 1 ! 1 


1 1 1 1 

1 1 1 1 
1 1 1 1 
1 t 1 1 

! 1 i ■/ 


\/ 


o 



1 

>- 




c 

7 

f 

f 


/ 


1 :/ 
1 :/!/ 


/ 
/ 


/I/ 1/1/ i/ 
1/1 1 1 

A /\/\f 1/ 
/j 1/!// 


\l\ j / 1 / 

/ 1/ i /|/ !^ 

l\l\ !/ i 


/ 1 / 1 ^ i / i / 
/ 1 1/1/1/ 

|/; 1/1/ 

/l/i/l i 
/!/i/i/l 


> 1^ 

/ ! / 


7 
V 

'7 




1 


i 1 ' 

1 ' 1 
> 1 1 
■ 1 

1 I 

1 , 1 


/ 


/ 1/ 1' 1/ 1/ 

: ! ! ! 


[ l/i 1^ 

/I 1/1/ 1/ 
/ 1/1/1/1/ 

1 ' ' ' 


; 1/ 1/ |/ 
/ /!/i/i/ 

/!/;/!/;/ 

ill! 
1 1 1 1 


' 1/ 
/ 1/ 
/ j/ 

' V 


J. ^ 





I 


X } J 


h 


r J- (>{>(, 


^ W 7/ 


^ / ^ M 


'if 


1 i fn 


5/W 



FIGURE 2 

SHOWING FORM IN WHICH DATA WERE TABULATED 

FROM TEST I 



30 Standardized Exercises in History 

These tabulations of school by school were made on cross sec- 
tion paper Wx2V' ruled 5 spaces to the inch. Every fifth line 
Mas drawn heavier to assist the eye in keeping track of certain points 
in counting. Along the top of each sheet were written the num- 
bers corresponding to the numbering of the papers handed in by 
the pupils examined. There was thus in each instance as many col- 
umns needed as there were pupils or papers. Every fifth number 
was written more heavily or in red ink. 

Along the side at the left were written the numbers of the 
exercises corresponding to each exercise of the test. In tabulating 
the scores for Test I, 50 horizontal lines were needed for each 
school. Every fifth niunber was written more heavily or in red ink 
for convenience in counting. There were needed 60 of such large 
sheets. They were held together by strong steel clips, and pro- 
tected by heavy flexible card board. 

Errors only were tabulated. If a pupil made but one error on 
the test, only one mark was placed in the column assigned to his 
paper. The other spaces of that column were left blank. An error 
was indicated by a straight line drawn diagonally at the intersection 
of the proper vertical and horizontal spaces. The sum of the num- 
ber of marks for each column gives the total number of errors or 
score for each pupil. The sum of the number of marks for each 
row gives the total number of errors made by all the pupils of the 
school on each exercise. The grand totals of the sums at the right 
of the page and along the bottom are equivalent. These totals 
serve as a check in counting. 

Another convenience in checking was found worth while when 
the papers were arranged in order of the scores made, beginning at 
the smallest number of errors. The papers were then numbered 
consecutively to correspond with the numbers at the top of the large 
tabulating sheet. In addition to this advantage the median score 
for any school was found almost at a glance. 

As a check on the accuracy in tabulating, the number of errors 
made by each pupil was counted as fast as each paper was disposed 
of. This total equals the total number of errors recorded at the 
top of the paper, or the pupil's score on his own paper. Since these 
two numbers check all along, the sum of the errors made by all 



Standardizing the Material 31 

pupils equals the sum of the errors on each of the exercises, the former 
number serving as the standard. 

The tabulation thus is quite rapid and easy so long as the work 
is close to the numbering at the left of the sheet. As the record 
of each pupil moves to the right and farther away, the eye strain is 
increased. The writer overcame this by spacing and numbering the 
plain side of an ordinary school ruler in duplicate of the column of 
numbers representing the exercises at the left of the large tabulating 
sheet. The numbers were written in red ink for prominence and 
durability. This ruler was moved along column by column right 
at the space provided for each pupil's record. If certain guiding 
lines for the ends of the ruler are observed even the tendency to 
errors is slight and the eye strain is reduced to a minimum. Only 
after the discovery of these useful devices in tabulating was it pos- 
sible to dispose of the routine work rapidly and easily. 



CHAPTER III 

TENTATIVE SCALES IN THE FORM OF LIST X 

AND LIST Y 

From the results of Test II and Test III two tentative scales 
were made out, and administered as List X and List Y. Exercise 
number 1 in List Y was devised and purposedly added with the 
idea of trying once more to find an exercise so easy that all pupils 
could work it successfully. This exercise reads, "The most com- 
monly spoken language of early New England was Dutch, English, 
French, Scandinavian, Spanish." Exercise number 14 in List X was 
added with the purpose of seeing if it would turn out to be a diffi- 
cult one. This exercise reads, "The gap in location, age, and ideals 
between the Northern and Southern colonies was bridged over by 
the founding of Del., Md., N. J., N. Y., Pa." The rest of the 
content of these two tests is given in full in this chapter. The rank- 
ing in difficulty was the chief consideration in making the selection 
for these lists. In case of alternatives, frequency of use in later 
periods, or social usage determined the choice. 

The method of administering did not differ in principle from 
the method used in the preceding experiments but it was written 
out more in detail for the benefit of the supervisors who assisted the 
writer in gathering data. The chief purpose in writing up these 
detailed directions was to try out the plan devised in the preliminary 
work whereby pupils could assist in marking and correcting their 
own papers. A copy of these directions for administering the exer- 
cises to pupils and for scoring the papers is here given: 

EXERCISES IN UNITED STATES HISTORY 
Directions to Teachers for administering Lists X and Y: 

1. Place upon the blackboard: 

(a) Columbus discovered Africa, America, Asia, Australia. 

(b) The Revolutionary War began in the year 176S, 1765, 1775, 1776. 

(c) The London Company was formed in England, France, Holland, 
Italy. 

2. Say to Pupils: "Here are some exercises in United States History. 

They are turned into historical facts when a line is drawn around 
the correct one of the several suggested answers." Draw a line 

33 



34 Standardized Exercises in History 

around America, 1775, and England in the examples, and read the 
exercises as correct historical facts. 

3. "I shall pass around exercises similar to these which are worked in 
the same way. 

4-. "Try to work out each exercise whether you are sure of the correct an- 
swer or not. 

5. "Take as much time as you need, but no more. 

6. "When all have finished working I shall read the correct answers and 

you may correct your own papers. 

7. "Mark only the numbers of the exercises incorrectly worked. Use an 

'Y ' " 

8. After pupils have made the necessary corrections, direct them to ex- 

change papers and keep them. 

9. Read over the correct answers again and have the pupils check over 

the papers to detect any errors or omissions in the first correction. 

10. Place on the blackboard the numbers corresponding to the exercises. 

11. Say to Pupils: "Look on the papers you now have and see if a mis- 

take was made in working out the first exercise, that is No. 1. 
If so, please raise your hand and hold it up until I take the count." 

12. Count the number of hands raised and place the sum opposite No. \. 

13. Repeat the process until the number of errors made on each exercise 

has been correctly determined. 

14. Copy the results for each set of exercises and send them and the papers 

to Sturgiss B. Davis, School of Education, University of Pittsburgh. 

List X 

EXERCISES IN UNITED STATES HISTORY 
Colonial Period 

Draw a line around the correct suggested answer in each Exercise. 

1. When the earliest colonists arrived in America they found the land T 

already inhabited by the Chinese, Cossacks, Indians, Malays, 
White men. 

2. The "Mayflower Compact" was drawn up by the Baptists, 2 

Huguenots, Methodists, Pilgrims, Quakers. 

3. The "Thirteen Original Colonies" were at one time all under the 3 

political control of England, France, Holland, Spain, Sweden. 

4. The most common means of travel used bj- the earliest settlers was 4 

by aeroplane, railroad, steamboat, trolley-line, ivalking. 

5. One of the turning points in our Colonial History was marked by 5 

the capture of Acadia, Deer field, Louisburg, Port Royal, Quebec 
in the French and Indian War. 



Tentative Scales in the Form of Lists X and Y 35 

6. The political control of the Dutch in America came to an end when 6 

Governor Stuyvesant surrendered to the Engtisli, Frencli, 
Germans, Spaniards, Sivedes. 

7. The Quakers founded the colony of Ga., Mass., N. II., Pa., S. C. 7 

8. The first attempt of the English to colonize in America was in what 8 

was later called the colony of Conn., Del., Mass., Pa., Fa. 

9. The period of settlement for most of the colonies took place in the cen- 9 

tury beginning with UOO, 1500, J 600, 1700, 1800. 

10. Local business affairs in the New England settlements were generally 10 

managed by the governor, king, mayor, minister, toiv?i-meeling. 

11. The territorial claims of the English in America were established by 11 

the treaty of Aix-la-Cliapelle, America, Paris, Rysivick, Utrec/it. 

12. The strict Puritan Church of early New England gradually gave 12 

way to what has since become known as the Baptist, Congre- 
gationalist, Lutheran, Methodist, Spiritualist church. 

13. The first written constitution in the colonies springing from the 13 

people and creating a government was the colony of Conn., Del., 
Mass., N. J., Fa. 

14. The gap in location, age, and ideals between the Northern and 14 

Southern Colonies was bridged over by the founding of Del., 
Md., N. J., N. Y., Pa. 

Put number of errors here 

List Y 

EXERCISES IN UNITED STATES HISTORY 
Colonial Period 

Draw a line around the correct suggested answer in each Exercise. 

1. The most commonly spoken language of early New England was 1 

Dutch, English, French, Scandinavian, Spanish. 

2. Among the many hardships of the time were the unexpected attacks 2 

by Arabian, Chinese, Cossack, Indian, Negro ivarriors. 

3. Every one of the colonies was founded by men from Africa, Asia, 3 

Australia, Europe, South America. 

4. By the time the "Mayflower" had arrived at Plymouth Rock It had 4 

crossed the Antarctic, Arctic, Atlantic, Indian, Pacific ocean. 

5. Under the "Treaty Elm" William Penn made a lasting peace with 5 

the Canadians, Huguenots, Indians, Negroes, Puritans. 

6. The light at night in the early homes came from candles, electricity, 6 

gas, kerosene, phosphorus. 

7. The Colonists and English soldiers were assisted in the French and 7 

Indian War by the loyalty of the Iroquois, Moliegan, Narragan- 
sttt, Pequot, W ampanoag Indian tribe. 



36 Stnndtirdiztd Exercises in History 

8. To stop the movement of the Colonists westward the French soldiers 8 

built a line of forts in western Conn., Mass., Md., N. J., Pa. 

9. More lasting and prominent political events took place in Baltimore, 9 

Boston, Charleston, Neia York, Pliiladelpliia than in any other 
one city. 

10. The colony of South Carolina was noted for the very early settlement 10 

of the Baptists, Catholics, Huguenots, Lutherans, Methodists, in 
the founding of Charleston. 

11. The last colony was founded in the 15th, 16th, 17tli, 18th, 11 

19th century. 

12. "Braddock's Defeat" occurred in the First, Second, Third, Fourth 12 

French and Indian, or Inter-Colonial War. 

13. The "United Colonies of New England" was formed in 1634, 1643, 13 

1683, 1733, 1774. 

14. Lord Delaware lived in England nearly all the time he was governor 14 

of Conn., Del., N. J.. R. I., Va. 
Your teacher will read the correct answers. Mark any Exercises missed 
on the left side with an "X." Count the number of Exercises missed and 
place the sum here. 

Number missed 

These lists were submitted to 1,564 pupils early in 1920. 
Many of the pupils were in schools of the city of Pittsburgh where 
a double promotion system is in practice. The pupils tested were in 
most cases nearly ready for the mid-year promotion to high school. 
Some of the schools were in boroughs near the city where there 
was also the mid-year promotion system. Sixteen schools in widely 
separated sections were chosen. All the work was done by the 
writer or by competent supervisors. 

The consistency of the results from these 1,564 pupils when 
compared with the previous data on the same exercises convinced 
the writer that the many long hours he spent in scoring and tabu- 
lating the results of Tests I and II were expended unnecessarily. 
If all the steps are well planned in advance and if the teachers help 
to guard against irregularities among pupils, and count the show of 
hands for errors, all the necessary data for weighting questions may 
be had right in the classroom. In one instance 260 pupils were 
gathered in an assembly room. They were seated in groups in 
alternate seats. Each teacher was detailed to give an account of 



Tentative Scales in the Form of Lists X and Y 37 

her own group. With the assistance of a blackboard and carefully 
laid plans, all these pupils were examined ; the papers corrected and 
checked ; and the number of errors on each exercise tabulated in 
little more than a double classroom period. Tables III and IV 
show how the data were tabulated as saved from school to school. 
In these tables are designated the names of the different schools 
and the number of pupils tested in each one; the number of each 
exercise, and the total number of errors made by the 1,564 pupils 
examined on each of the exercises. 

All that was necessary in addition to this data was to find 
what per cent, of all the pupils examined missed each of the exer- 
cises in Lists X and Y. As the data from Lists X and Y could be 
supplemented by the data from the preliminary work on Tests II 
and III, this was done. The number of pupils tested was thus 
brought up to either 2,064 or 2,266 for each exercise, except in the 
case of number 13 in List Y where the grand total was 2,766. The 
values given in Tables V and VI are based upon these numbers. 
Tables VII and VIII are added to show how closely the ranking 
of the exercises conforms to the values found from the preliminary 
work on Tests II and III, and that the remaining exercises in Test 
II and III might be used at will. The correlations are almost per- 
fect by Spearman's formula. 

Before concluding this chapter it should be stated that the 
author's prior estimate of the ease or difficulty of an exercise was 
only the roughest kind of an approximation. What was supposed 
to be a very easy exercise ranked fifth in difficulty. Other similar 
instances appeared throughout the study. 





38 








Standardized Exercises in 


History 












■-/^'^'o 1 














tal 

upils 
choo 






NUMBER OF EACH EXERCISE 


t-' 






: 5-z 
























1 








. o 


>^ 


H- 
W 


to 


^ 


o 


vO 


00 


^ 


ON 


fj\ 


*>■ 


M 


to 


H- 


X 






: S-°. 






















1 














KJ 










t_^ 




to 
















> 








s 


Cn 


•-J 


to 


lo 


'-' 


(-rl 


O 


O 


Ol 




^^ 


■^ 




^ 










§ 


OO 
oo 


ON 


CN 


Oi 


On 


OO 


Ln 


o 


OJ 




to 


O 


OO 


o 


td 


-5 
> 




































r 












■-J 


'_n 


OJ 


f-rt 


.^ 


_ 


_ 








o 


O 




oo 


^ 


H~^ 


o 


to 


to 


OO 


H- * 


^J 


■^ 






ON 


^ ' 












































1—" 






































M 




o 








'DC 


~^ 


~s 


'^f\ 


o-l 


^^ 


_ 










o 




1 




f,^ 


to 




OJ 


CN 


o 


4- 


OJ 


to 
















cn 




































ffi 






































'O 


■^ 
■M 


OO 


g 


OO 


OO. 

O 


OJ 


to 


o 


OJ 
O-l 


to 

O 


^ 


On 


4^ 


NO 


m 




O 




































'Z 












4^ 


,_i 


, 1 


OJ 


KJ 


1 — ' 












■-n 









to 


^J 




o-> 


-^ 


ON 


OO 


OJ 


'-JO 


4- 


^J 


4^ 


4^ 




^ ' 










































o 














Oi 






_ 














O 








ON 


(^ 


4^ 


O 


4^ 


^J 


'"' 


4- 


--J 


NO 


4^, - 








(73 

o 

o 
o 


< 






VO 


to 


^ 


ot 


4^ 


OO 


H- 


to 


CO 


^4 


to 


O 


O 


o 


X 


> 


































> 




o 


0>i 


^ 


OO CO 


ON 


On 


~-J 


-- 


OJ 


On 


- 


o 


o 


- 


' ' 




> 






































, . 


,_i 


1 . , I 
































OJ 




'O to 


^J 


ON 


O 


f^n 


Oj 


On 




»— » 












oo 


On 


^J 


4- ■— 


NO 


IJl 


-J 


^ 


to 


"" 


































1 










n 








to 


to to to 


, , 


, t 






1 






?^ 




o 




4^ 




ON 


■vO ^-^ ^J 


Ln 


■M 


NO 


OJ 


OO 1 O O 




^^ 






a 




















1 1 


















OO f^Kj to 


,_, 


^^ 




i - i 1 






r 






O 




OO 


On 'CC NO 


i-K> 


Ln 


OJ 


ON 


CO to to 


►f' 


•"^ 














1 


























^jO 


ON *- 1 4^ 


4^ 


OJ 


OO 


to 








K 






o 


^I 


OO 


t^ ~v} OO 


o 


CN 


to 


OD 


On OO On 




"^ 






n 


































to 

o 


NO 


H- OO to 


o 


l-ri 


OO 


4^ 


ON H- 


^ 




z 




o- 




o 


to 


4^ 


NO ^-n NO 


NO 


'J-l 


t— * 


--J 


1— ' Ol CO 


to 
























1 












o 




























f 








1 , 


OJ 1— • to 


,__, 














o 








4^ 


ON 


c:^ 


— 4^ to 


•" 


NO 


ON 


Ol 


ON "-^l O 


o 


to 












A^ 


*■ 


On H- » ON 








^ 






Tl 






O 


<-n 


i^ 


to ^J 1— ' 


4- 


to 


o ^ 


On to O 


'"' 


o 




















1 




1 










E?§-s 








^O 


\o 


NO 


OO -^ 


'_n 'ji ►**- 


to 












M M?^ 






Cn 


KJ-, 


4^ 


00 


^-n (Ji 


to -J O 

O 4- O 






OO 


4^ 


>+>. 


*. 






rf^ 


ON 


o 


ON 


-J o 


O 


-vj 


to 


OO 




4^ 


5; o o Z 












































1 


1 












n> =^^ 





Tentative Scales in the Form of Lists X and Y 



39 





criifl^ 


























































al 

pils 

lool 






NUMBER OF EACH EXERCISE 






3 






H3-^ 


































O o 


^^ 


^ 


^ 


^4 


^.^ 




















Hf' 






^ M' 


4^ 


M 


K) 


Nrt 


o 


nO 


oc 


<I 


ON 


yi 


4^ 


M 


K) 


^•^ 








n » 




































Q. 2- O 




































*-' —*-. 




































tsj 






































o 


4^ 


o.i 


i^ 


to 


to 


to 


















> 








■o 


tx> 


Ln 


t-n 


to 


to 


oj 


oo 


oo 


OO 


to 


O 


^ 


O 


to 










































H 




o 


'-ri 


CC' 


CO 


i-n 


4^ 


4- 


'-^ 


1 — 1 


H-1 


4- 










td 




> 




+- 


ON 


i-Ki 


•"^ 


Ui 


OJ 


4- 


4- 


^J 


4- 


4- 


' — ' 


^^ 


4- 


O 






ca 




































m 




yo 


•^ 


•^ 


C~> 


On 


ON 


4- 


4- 


to 


to 


i__i 




,_^ 






O 








'/o 


4- 


^J 




OO 


o 


Oj 


to 


to 


O 


O 


NO 


O 


Ol 


O 




■^ 




O 


^O 


CO 


-~J 


4^ 


~-4 


to 


to 


to 


to 












D 


1 




'^-j 


OJ 


o 


o 


'00 


to 


-~4 


On 


NO 


O 


^^ 


to 


oo 


ON 


O 




K 






































O 




;^ 


vo 


NO 


-4 


oo 


NO 


4^ 


Oi 


4- 


'^aJ 


^^ 




,_, 




, . 


M 




53 




o 


to 


NO 


NO 


ON 


'"* 


'~' 


4^ 


to 


(^ 


*"^ 


NO 


^4 


"^ 


ON 






2 






•> 


<J1 


*- 


4>- 


4- 


4- 


OJ 


Oi 
















^ 






■sj 


"^ 


NO 


oo 


OJ 


to 


' ' 


' — ' 


oo 


'-" 


4- 


(-n 




o 


Ol 






a 






































o 




4^ 


^>-J 


'-Kj 


'.JJ 


O-J 


^^ 




to 


^- 














O 








'-^ 


-^ 


to 


4- 


•"' 


oo 


'-'■' 


4- 


4- 


On 


ON 


^ 


to 


to 






a 






































hJ 

:> 


C^ 


ON 


'-n 


NO 


ON 


GO 


■^ 


NO 


O 


O 


1^ 


O 






S^ 


(/2 

n 


> 


































K 


> 




i~o 












^ 




















53 




o 


4^ 


KJ 


<-Ki 


OJ 




to 


-~4 


'"' 


*~' 


^^ 


o 


to 


"— ' 


O 




C/2 


> 










































to 


|_-4 


1.^ 


H-4 


t_> 




























-ri 


O 


oo 


o-i 


to 


to 


^4 


OO 


ON 


»— ' 


h— ' 








H-* 


' 1 




pi 




7D 


4^ 


oo 


OJ 


1^ 


"^ 


to 


^>J 


On 


NO 


'"' 


t-n 


to 


ON 


.OO 






O 








































































O 




-t^ 


4^ 


t>i 


to 


to 


to 


H^ 


)— * 
















t^ 




a 




4>- 


O 


V>i 


ON 


00 


■^ 


OO 


t^ 


Oi 


4- 


O 


O 


O 


o 


O 






































M 




4^ 


4- 


'^^ 


'-^ 


to 


to 


to 


^1 


„- 














H-l 




a 




:^ 




NO 


to 


-4 


'■^ 


4- 


4- 


+- 


'~' 


^ 


o 


'— 


^ 


to 






Cd 




































K! 








^J 


u-i 


Ln 


OJ 


'.-Ki 


Oi 


to 












1 . 


•^ 








o 


O 




4^ 


OJ 


ON 


*- 


OO 


<j-i 


Ln 


t-K) 


NO 


^ 


4^ 


Ut 






O 






hO 


to 


^^ 


^ 


_ 




^^ 


















X 

o 




3> 


I_Kl 


O 


oo 


to 


--4 


NO 


O 


*- 


'-►i 


to 


to 


1—1 




4->. 


^; 






o 


O 


NO 


to 


Ln 


Oi 


1—' 


On 


(-n 


■-4 


4^ 


on 


NO 


ON 


ON 




o 














































































._^ 


K> 


to 


>.^ 


to 


to 




















o 








■4^ 


^ 


oo 


I^ 


^^ 


O 


ON 




^^ 


to 


^^ 


'"' 


•~* 


" 


^^ 










Ln 


t-n 


Ln 


On 


to 


^ 


to 


^ 














>TJ 








o 


^I 


'~' 


■"" 


O 


^^ 


4^ 


^^ 


O 


as 


4^ 


"^ 


^^ 


o 


to 








































Total 
of Err 
OT Ea 
Exerc 














NO 


^4 


•^ 


4^ 


Ol 


Oi 


















On 




t-n 


o 




NO 


t-n 


NO 


1—* 


to 


-o 


lO 


OO 


^4 


'-^ 


4- 






4^ 




'-n 


^4 


O 


<-n 


4^ 


ON 


to 


'--n 


•^ 


^^ 


oo 


to 


oo 


O 






































«-o o z 






































"^ =" ^ o 





40 



Standardized Exercises i?i History 



TABLE V 
TOTAL NUMBER OF ERRORS MADE ON EACH EXERCISE IN LIST X 



Exercise 


Number 


Same 


Same 


Grand 


Percentage 


Sigma 


Number 


of Errors 


Exercise 
in Test II 


Exercise 
in Test III 


Total 


Value 


Value 


1 


44 


6 




50 


2.42 


.63 


2 


49 


13 




62 


3.00 


.70 


3 


48 




36 


84 


3.70 


•79 


4 


82 




64 


146 


6.44 


1,03 


5 


257 






257 


16.42 


1.55 


6 


230 


84 




314 


15.20 


1.50 


7 


400 




183 


583 


25.71 


1.87 


8 


574 


173 




747 


36.15 


2.16 


Q 


520 


200 




720 


34.85 


2.13 


10 


750 




428 


1178 


51.95 


2.53 


11 


857 




285 


1142 


50.36 


2.50 


12 


986 




571 


1557 


68.66 


2.97 


13 


940 


347 


493 


1780 


64.44 


2.85 


14 


956 






956 


61.09 


2.77 


Total No. 


of Pupils 
1564 


taking 
500 


each test 
702 









TABLE VI 
TOTAL NUMBER OF ERRORS MADE ON EACH EXERCISE IN LIST Y 



Exercise 


Number 


Same 


Same 


Grand 


Percentage 


Sigma 


Number 


of Errors 


Exercises 
in Test II 


Exercises 
in Test III 


Total 


Value 


Value 


1 


140 






140 


8.95 


1.19 


2 


38 






27 


65 


2.87 


.69 


3 


72 






51 


123 


5.42 


.95 


4 


88 






60 


148 


6.53 


1.04 


5 


123 








123 


7.86 


1.13 


6 


177 






i93 


370 


16.32 


1.54 


7 


325 






223 


548 


24.16 


1.82 


8 


512 


i67 




679 


32.86 


2.07 


9 


496 




384 


880 


38.81 


2.23 


10 


754 






754 


48.18 


2.47 


11 


795 




504 


1299 


57.29 


2.67 


12 


950 


314 




1260 


60.98 


2.76 


13 


1107 




487 


1594 


70.30 


3.02 


14 


1155 


363 


550 


2068 


74.86 


3.15 


Total No. 


of Pupils 
1564 


taking 
500 


each test 
702 









Tentative Scales in the Form of Lists X and Y 



41 



TABLE \"I1 
RANK OF EXERCISES IN LIST X IN SECOND TRIAL 



Rank and 
No. of 

the 
Exercise 



Percent 

of all 

Pupils 

Missing 



Sigma 
Value 



Rank in 

Second 

Trial 



Abstract of Contents 
of Each Exercise 



6 
7 
8 
9 

in 
11 

12 

13 
U 




Indians here before the white men came 
"Mayflower Compact" for the Pilgrim 

Fathers 
England controlled 13 original colonies 
Walking most common means of travel 
Capture of Quebec turning point 
Stuyvesant surrendered to the English 
Quakers founded Pennsylvania 
Va. colony lirst English settlement 
Period of settlement from 1600 to 17fX) 

mostly 
Townmeeting managed business affairs 
Treat}- of Paris established English 

claims 
Puritan cluirch represented by Con- 

gregatlonalist 
First written constitution In Conn. 
Pa. bridged gap between North and 

South 



Correlation by Spearman's "Footrule" is .98. 



TABLE VIII 
RANK OF EXERCISES IN LIST Y IN SECOND TRIAL 



Rank and 
No. of 

the 
Exercise 



Percent 
of all 
pupils 

Missing 




Rank In ] 
Second j 
Trial : 



Abstract of Contents of 
Each Exercise 



6 

7 

8 

9 

10 

11 

12 

13 

14 



1.19 

.63 
.88 
.97 

1.13 



V 
I 

II 
III 

IV 



11.31 


1.32 


VI 


20,77 


1.71 


VII 


32.72 


2.07 


IX 


31.69 


2.04 


VIII 


48.18 


2.47 


X 


50.80 


2.50 


XI 


60.70 


2.76 


XII 


70.74 


3.03 


XIII 


73 . 80 


3.12 


XIV 



English Language most common 
Indian attacks among the hardships 
Men from Europe fotmded the colonies 
The Mayflower crossed the Atlantic 

Ocean 
The Treaty Elm marked peace with 

Indians 
Candles furnished the light at night 
Iroquois assisted the English soldiers 
French built a line of forts in Pa. 
Philadelphia a political center 
Huguenots settled Charleston 
Last colony founded in 18th century 
Braddocks Defeat In French and Indian 

War 
"United Colonies of New England," 

in 1643 
Lord Delaware, Governor of Va. 



Correlation by Spearman's "Footrule" is .97. 



CHAPTER IV 

WEIGHTING THE RESULTS OF THE DATA ON EACH 

EXERCISE 

Sec. 1. — Plan of Procedure. After tabulating the total number 
of errors made by all the pupils taking each exercise these numbers 
were transformed into percentages. The first question answered was : 
what per cent, of all the pupils examined failed in exercise number 
one, two, three, four, and so on for the whole list? 

The plan adopted for doing this was as follows: a blank book 
of cross section paper was procured which had more than twice as 
many pages as the number of exercises to be weighted. The right 
hand pages of the book were numbered consecutively in large figures 
in red ink. As many pages were numbered as there were exercises 
to be weighted. In a column at the left on these pages were listed 
the numbers of the large tabulation sheets in which the data from 
each school were tabulated. After each of these numbers was listed 
the total number of errors found for each exercise on the large tabu- 
lation sheet. The sum of these numbers taken sheet by sheet gave 
the total number of errors made on any given exercise. 

On the left hand pages of the book were made the necessary 
calculations to determine the weight of each exercise. These calcu- 
lations consisted in finding the percentages and then transmuting 
them into sigma values. When the data on the 1,250 pupils tested 
in 1918 on Test I were utilized, advantage was taken of the con- 
venient quotient given when 100 is divided by 1250. This is .08 
and was used as a constant multiplier in finding the percentage of 
errors made on each exercise. The work was further facilitated by 
the use of Crelle's Calculating Tables. An allustration is offered 
to save further description. On a certain exercise, X pupils failed 
out of the 1,250 examined. Required, to find the percentage of 
failure in this instance. 

X 100 100 X 

X = X — = 8X% 

1250 1 1250 1 

When the equation is stated in this form it is evident that the frac- 
tion 100/1250 is a constant in each calculation, and the variable 

43 



44 Standardized Exercises in History 

is always the number of errors made on each exercise. Hence the 
calculation in each instance is reduced to multiplying .08 through 
means of a calculating table by the total number of errors made on 
each exercise. 

The next step was the transmutation of these percentage values 
into sigma values. This was done by mean.s of the tables prepared 
for such a purpose by Rugg. Both Tables V and VI in this man- 
ual were considered but Table V was chosen because it is built upon 
the usual five divisions of the base line of the Normal Curve of 
Frequency. It is also accurate enough for the purposes of this study. 

No attempt is made here to explain the Normal Curve of Fre- 
quency or its use in finding the absolute ease or difficulty of ques- 
tions. Its use is now widely recognized and accurate descriptions 
and explanations are available to students of educational problems. 

Many investigators of educational problems have used the P. E. 
values in weighting questions. Sigma values were used in this study 
because these new tables of Rugg are more convenient to read and 
the values may be found directly regardless of the median point of 
reference. The sigma unit value was also used because it is a nat- 
ural function of the Normal Curve and is widely used by students 
in Riology, Sociology, Y\griculture, and Actuarial Work. 

For the sake of comparison, however, the P. E. values of Test 1 
were calculated and compared with the sigma values. In order to 
get these P. E. values it was necessary to assume a point of reference, 
since no exercise was found which all pupils could answer. This 
point of reference was — 4.3 P. E. from the mean ordinate. It was 
derived from the first exercise which 98.68% of the pupils worked 
out correctly. Table XLVII in Buckingham's Study of "Spelling 
Ability" was used in calcidating the P. E. values. Table IX of this 
study shows the comparisons in full. 

Sec. 2. — Tests for the Reliability of the Data. Several means 
of checking on the reliability of data were considered before the 
values here presented were considered valid. These means are all 
discussed in books on Statistical Method and need only to be men- 
tioned here. 

The curve of errors shown in Fig. I was plotted and compared 
with the normal curve drawn from the same median ordinate. 



Weighting the Results 

TABLE IX 
COMPARISON OF P. E. AND SIGMA VALUES 



45 



No. of 


Percent 


Difference 


P. E. 


Dist. in 


P. E. 1 


Sigma 


I'lzercise 


Correct 


between 50';'v 


Values 


P. E. above 


Value 


Value 






and Score 




Zero Point 






1 


98 68 


-48.68 


— 3.3 


1.00 


10 


4 


2 


68 00 


-18.00 


- .693 


3 . 607 


36 


20 


3 


64.08 


-14.08 


- .535 


3 . 765 


37 


22 


4 


63.08 


-13.08 


- .49f, 


4.104 


41 


24 


5 


55.20 


- 5.20 


- .194 


4 . 406 


44 


24 


b 


55.84 


- 5.84 


- .216 


4.084 


40 


24 


7 


65 . 4 


-15 4 


- 588 


3.712 


37 


21 


s 


20.04 


+ 29.96 


+ 1.243 


5.543 


5> 


33 


9 


74.96 


-24.96 


- 996 


3.004 


30 


19 


10 


52.20 


- 2.20 


- .082 


4.218 


42 


25 


11 


55.04 


- 5 04 


- .187 


4.113 


41 


24 


12 


85 . 64 


-35.64 


-1.576 


2 . 724 


27 


15 


13 


19.04 


+ .30.96 


+ 1,296 


5 . 596 


x5 + 


34 


14 


81.20 


-31.20 


-1.313 


3 . 287 


29 


16 


15 


29 . 84 


+ 20.1'. 


+ .782 


5.082 


50 


30 


16 


44 . 88 


+ 5.12 


+ .187 


4.487 


45 


26 


17 


38.72 


+ 11.28 


+ .426 


4.726 


47 


28 


IS 


59.08 


- 9.08 


- .341 


3.959 


39 


23 


19 


68.44 


-18.44 


- .710 


3 . 590 


35 


20 


20 


47.36 


+ 2.64 


+ .097 


4.397 


44 


26 


21 


60.20 


-10.2 


- .383 


3.019 


39 


23 


11 


87.64 


-37.64 


-1.713 


2 587 


25 + 


14 


23 


74.64 


-18.64 


- .719 


3.581 


--' + 


19 


24 


86.84 


-36.84 


- 1 . 656 


2 . 644 


26 


14 


25 


90.16 


-40.16 


-1.980 


2.320 


23 


12 


26 


29. 2 


+ 20.68 


+ .807 


5.107 


51 


30 


27 


38.84 


+ 11.16 


+ .422 


4.722 


47 


•28- 


1>^ 


71.52 


-21.52 


- .842 


3.458 


34 


1'^+ 


29 


55 . 76 


- 5.96 


- .213 


4.087 


41- 


24- 


30 


39.80 


+ 10.20 


+ .383 


4.683 


47 


27 


31 


58.28 


- 8.28 


- .311 


3.989 


40- 


23 


12 


55.40 


- 5.4 


- .201 


4.099 


40+ 


24 


33 


18.92 


+ 31.08 


+ 1..307 


=^ 607 


56 


34 


34 


37.56 


+ 12.44 


+ .468 


4.768 


48 


28 


35 


56.92 


- 6.92 


- .258 


4.042 


40 


23 


36 


68.16 


-18.16 


- .698 


3.602 


36 


20 


37 


20.08 


+ 29.92 


+ 1.243 


5.543 


55 


33 


38 


96.80 


-46.80 


-2.746 


1.554 


15 


7 


39 


42.92 


+ 7.08 


+ .261 


4.561 


45 


26+ 


40 


72.00 


-22.00 


- .864 


3.436 


34 


19 


41 


68.52 


-18.52 


- .714 


3.586 


36- 


20 


42 


49.28 


+ .72 


+ .026 


4.326 


43 


25 


43 


86.88 


-36.88 


- 1 . 663 


2.637 


26 


14 


44 


82.84 


-32.84 


-1.403 


2.897 


28 + 


16- 


45 


87.12 


-37.12 


-1.677 


2.623 


26 


14 


46 


59.32 


- 9.32 


- .349 


3.951 


39 


23 


47 


53.68 


- 3.68 


- .172 


4.128 


41 


24 


48 


21.76 


+28.24 


+ 1.155 


5.455 


54 


32 + 


49 


60.80 


-10.80 


- .407 


3.895 


39 


22 


50 


98.30 


-48.15 


-3.077 


1.223 


12 


5 



46 Standardized Exercises in History 

The same curve of errors was smoothed successively until no 
appreciable change was evident. 

In both these instances the theoretical curve did not differ de- 
cidedly from the real curve obtained from the responses of children. 

The check on the reliability of the data which was most used 
was the consistency shown in the percentage of errors pupils made 
on each exercise from school to school. In the case of each test there 
came a point where the addition of more data from pupils made 
little or no change. When that point was reached, no further data 
uere taken. 

Another test of the reliability of the data, though an indirect 
one, was the constant tendency revealed in the results from each 
school. This tendency was the wide variability of the achievements 
of individual pupils. Since the pupils who took the test were all 
the most advanced ones in the elementary school such variability was 
not expected, unless it was due to inherent mental differences which 
the history tests brought out. These statements are supported by 
Table X which shows the wide range of errors for each of the 58 
schools taking Test I. 

If the variability was due to the inherent capacity of pupils 
to master the formal requirement of the school in United States 
History, the scores in these history exercises should indicate corre- 
lation with the scores of a recognized intelligence test. Table XI 
was prepared from the scores of 203 pupils who had taken the 
Pressey Intelligence Test and Test III of the history exercises. The 
scores are not altogether comparable since the intelligence test is 
really a composite of several tests, each intended to test a different 
mental ability. The history test on the other hand invokes prob- 
ably largely the factors of selective judgment and recall. How- 
ever, it was the best data obtainable at the time on this point. 

An examination of the correlation table made up of these two 
classes of scores shows that they produced much the same general 
effect. In both tests the variability was marked, but in general th.' 
brighter pupils made the better history scores. The median score 
in the intelligence test was 140.6. Only twelve pupils ranking 
below this median score in intelligence made a high score in the 
history test. Both tests also showed large individual differences in 



Weightirig the Results 



47 



TABLE X 
RANGE OF ERRORS MADE BY INDIVIDUAL PUPILS 
IN THE SCHOOLS TAKING TEST 1 



Schools 


Range 


Schools 


Range 


1 


1-27 


XXX 


13-33 


II 


4-35 


XXXI 


5-27 


III 


8-34 


XXXI 1 


10-36 


IV 


11-33 


XXXIII 


2-28 


V 


17-31 


XXX IV 


7-28 


VI 


9-41 


XXXV 


4-33 


VII 


6-32 


XXXVI 


1-30 


VIII 


&-37 


XXXVII 


8-40 


IX 


5-32 


XXXVI 11 


11-31 


X 


^31 


XXXIX 


11-31 


XI 


5-30 


XL 


5-29 


XII 


7-26 


XLI 


7-29 


XIII 


5-25 


XLII 


9-37 


X V 


4-29 


XLI II 


9-36 


XV 


5-27 


XLIV 


7-43 


XVI 


7-25 


XLV 


2-32 


XVII 


11-33 


XLVI 


S-32 


XVIII 


13-36 


XLVII 


7-35 


XIX 


5-29 


XLVI 1 1 


2-40 


XX 


7-27 


XLIX 


2-30 


XXI 


6^23 


L 


^31 


XXII 


7-31 


LI 


9-36 


XXIII 


8-32 


LII 


4-38 


XXIV 


9-36 


LIII 


12-36 


XXV 


17-40 


LIV 


1-18 


XXVI 


7-23 


LV 


7-35 


XXVII 


3-34 


LVI 


12-33 


XXVIII 


^38 


LVII 


3-36 


XXIX 


11-38 


LVIII 


16-37 



the pupils. These facts suggest that the material of the history test 
contained exercises, some of which were inherently easy and some 
inherently difficult. This last statement is consistent with the con- 
stant tendency revealed in the percentage of errors pupils made on 
certain exercises when the attainments of pupils were compared by 
schools. 

Having found the marked differences in the scores of pupils of 
the same school it was pertinent to find out whether the same char- 
acteristic would be found among the different schools themselves. 
The measure used for this test of reliability was the coefficient of 
dispersion based upon the semi-inter-quartile range and the median 
score found for each school taking Test I. Table XII shows these 



48 



Standardized Exercises in History 
POINTS IN PRESSY INTELLIGENCE TEST 



^^^3^"5^$^^^5^^5^^^ 




^ rl t>< k^*"^ C^<^ t-^tS f-^r^ *^<~< r^ '~< ^^^S ,5"^ 


f 




^\ \ \ \ \ \< 


i/i ^ 


1 










0- 


f 






>u :/; i 1/ 


/i/U 


^ 


2 


/ 








/3 


/D 






;l; ivi i^: 


^/iv 


X 


1 


3 


-^1/' 




/ 


ZL 


11 








o\^\^ 


^ 


7 


J- 


^a|/ 






0-/ 


IL 


1 




^W\^ 


c 


3 


/ 1 , 


/ 




a 


J3 






l\ 1/| l/l-^ 


/i^ ? 


^ 


/ 


•V 


^"' ''i 


/ 




^J 


i4 






'II, 
1 1 ' ' 


3i/;/^ 


^D 


7 


3 


J yi 






22- 


>S 






1 1 1 1 ' 
III]' 


/\3\ 


3 




^ 








/a 


// 






I'll, 
'III 


/I/; 






/ 


S/ 






y 


'7 








1 1 1 1 1 


/I I 






/ 


;/> 






3 


^ 






i /; 1 : i 


1 1 
i 1 








J ; 




^, 


/ 




1 


6 ^ %] z /o /^:^i/f;i2)Xr/i>27 /(, /o / j^ / 


/ 



Table XI. Corieliition table showing relationship between scores correct in 
Piessy Intelligence Test and Test III of the Exercises in Colonial History 



coefficients of dispersion for each of the 58 schools taking the test. 
The difference noted in the range is inconsequential when the facts 
are known about these two schools. The school in which the lowest 
coefficient was found is one whose scope is limited and the super- 
visor is a typical drill master. The school in which the highest 
coefficient appeared is located in an industrial center in the North- 
west. The pupils of this school were expecting to take a commer- 
cial course the following year in the high school. What is striking 
about this table however is the range of the middle-fifty per cent, 
of the cases. It is less than 10 points; Q is approximately 4.8; and 
the C. D. is but .22. These findings are consistent with the state- 
ment given out by the New Committee of Eight on courses in his- 
tory. This statement is to the effect that the present content of 
American History courses for the Seventh and Eighth Grades is the 
best standardized of all courses in history. The coefficient .22 con- 
forms remarkably well with the index of variability established by 
Courtis for his test in the four fundamentals of arithmetic. His 



Weighting the Results 49 

standard was derived in approximately the same way and is given 
as .20. No question has ever been raised either as to the content or 
value of the data upon which Courtis derived his standard. 

In concluding this topic of variability the writer feels justified 
in saying that the content of the history exercises is well recognized 
material ; and that the form of the exercises always reveals indi- 
vidual differences in the same class or school. When large groups 
are compared, however, the performance of pupils is approximately 
uniform. 

Table XII 

COEFFICIENTS OF DISPERSION DERIVED FROM THE SCORES OF 

THE 58 SCHOOLS TAKING TEST I 



Tidex 


Frequency 


9 


1 


11 


2 


12 


4 


13 


2 


14 


2 


IS 


4 


16 


2 


17 


4 


18 


1 


19 


1 


20 


3 


21 


4 


22 


4 


23 


6 


24 


1 


25 


4 


26 


2 


28 


1 


30 


4 


31 


1 


33 


1 


34 


2 


37 


1 


40 


1 




Total 58 



Range = 9 — 10 
Median = 21.7 
Q. = 4.8 
C. D. = .22 



50 Standardized Exercises in History 

Sec. 3. — The Reading Difficulty of the Exercises. In evaluating 
the reliability of pupils' responses the reading difficulty must be taken 
into consideration. Can the assumption be maintained that pupils 
may know history but because of the difficulty of the form of the 
test itself they do not know what is wanted of them? 

Some investigators have dismissed this question by saying that 
if pupils cannot read their school texts by the time they have reached 
the eighth grade, they are helpless so far as formal schooling goes. 
If the words used in the test are familiar to pupils, and if the con- 
struction of sentences are not unusual, the claim of these investigators 
is logical. 

In order to get direct evidence on the reading difficulty of Test 
I the writer submitted the exercises one by one to individual pupils. 
He went to the principal's office in two different schools and selected 
children from the eighth grade at random for personal interviews. 
In each instance the test was put before the pupil with the request 
that he read the exercise and simply state just what was to be done 
after reading it. No answers were required, and no comment made. 
The aim of the examiner was to get the pupil to think out loud. 

A case record was kept for each pupil by noting any peculiarity 
or difficulty in his responses. In no instance was there a single pupil 
who did not know exactly just what was wanted in each exercise. 
The words delusion, influential , indentured, and professional were 
not fully comprehended by some pupils when they were questioned 
after the examination, but in each case these same pupils gathered 
from the context of the exercise exactly what was wanted. These 
pupils had an impressionistic knowledge of these difficult words and 
recognized them sufficiently to use them in sentences. In the case 
of the exercise referring to the "witchcraft delusion occurring among 
the Puritans," some pupils passed over the word delusion and asso- 
ciated witchcraft with Puritans at once. Instances were found where 
the pupil could work out this exercise but could not use the word 
delusion readily in a sentence. The examination in this case was 
conducted somewhat after the manner of Terman in the vocabulary 
test used in his Stanford Revision of the Binet-Simon Test for Intelli- 
gence of School Children. 

Another instance in line with the above came out in respect 
to the word, indentured. One pupil remarked when questioned 



Weighting the Results 51 

about its use in the exercise, "You do not have to know what it 
means exactly ; servants and slaves gives you the idea of what is 
wanted." This boy knew the correct answer to the exercise also. 
These private interviews with pupils threw much direct light on the 
reading difficulty of these particular exercises and indicated that 
the mental process involved was associative memory to a marked 
extent. 

In order to get further data on the reading difficulty of this 
type of testing the scores of 241 eighth grade pupils who had taken 
the Monroe Reading Test, and Test III of the History exercises, 
were compared in a correlation table. Table XIII reveals at a 
glance that the degree of correspondence is low. As a matter of 
fact it is so low that it is negligible, being .04 by the Pearson formula. 

Test I could have been improved by simplifying the wording. 
However, it was thought wise to leave the exercises in the original 
form for the purpose of this study. All the words in the test are 
in use in standard text books and teachers have not criticized their 
inclusion. 



SCORES IN HISTORY TEST III 





i 7 /o n i^b/b''^/^ IL //// ^ 


// 


\'' 


X 




-^1 i :'i'i i : 


^ 


'9 


1 


\ 


J. 


^:3;3;3;/|/: 1 


Z. 1 


>1 




/ 


K 


f\j\3\v':X\/\/ , 


3 ^ 


x^ 


I 


f 


h 




37 


^3 


/ 


z 


z 


XT 


;-^ 


/ 


3 


y 


■3\y^;>ii\ 1 ; ;/ 


Xi 


%1 




X 


V 


;!//;>i-?:^; Uj/ 


^ o 


6X 




1 


/ 


yl3'^;r/i ; ; 


1 (. 


^ 






/ 


X\ i'2-|/^/l ; ! 


I 


i'i 






/ 


/I jii ; V\ ; 


W/ 




X 


X 


x: \v\/\ :\i 


1 / 


"7 




/_ 


3_ 


1 :v:^U: ; ;\ 


1 
/ A. 


^/f2i ¥?uv^ii/^ f i> ^ 


.A^ 


■t 1 



TABLE XIII 

Showing Correlation Between Scores in Monroe Reading Test and Test III 

in History 



52 Standardized Exercises in History 

Lists X and Y represent an improvement in the wording of 
exercises. Through practice one can readily become skilled in fram- 
ing exercises that meet all the standards set up in Chapter I. In 
general it may be offered that so long as the vocabulary of the exer- 
cises correspond to that of standard text books, this reading difficulty 
of words, or sentence structure, does not seem to be at fault nearly 
so much as the lack of purpose in reading. This was shown very 
definitely in a reading test prepared by the writer in w^hich para- 
graphs were taken verbatim from a school history which the pupils 
had used. 

Sec. 4. — The Element of Success by Chance. Formula. In 
addition to the reading difficulty it was suggested that some pupils 
were successful through the operation of a chance selection of the 
correct one of the several suggested answers. The tabulation of the 
scores on the large tabulation sheets where all the scores from a single 
school could be seen at a glance, gave no evidence of the operation 
of the law of chance. The difficult exercises were worked out only 
by the pupils who made good scores consistently throughout the test 
as a whole. When pupils were questioned individually regarding 
the reasons for the answers chosen, they always gave a plausible 
explanation even though their answers were incorrect. For the pur- 
poses of diagnosis it was desired that pupils select what they consid- 
ered to be the correct answer, and they were encouraged to try each 
one of the exercises. In no instance was there found evidence of 
wild, haphazard guessing at the correct answer among pupils who 
took the testing material for it was all more or less familiar to them. 
They thought they knew whether they did or not. 

In the event that it seemed advisable at any time to make cor- 
rections for the element of chance entering into the scores of pupils, 
a correction formula was developed. The formula was suggested 
by the one offered by McCall which he used in eliminating the 
effects of chance selection in his experiment with the "True and 
False" type of of examination. This is given in an article entitled 
"A New Kind of School Examination." The formula offered in 
this instance was: 'Tupil's Score = (number correct) — (number 
wrong." Starting with this formula it was developed so that any 
number of chance selections could be considered. Simultaneously 



Weighting the Results 53 

Buckingham did the same thing and expressed his formula as follows: 

"S = c — ^, where S is the score after account has been 
n — 1 

taken of the elements of chance ; c is the number of correct answers ; 
w is the number of wrong answers ; and n is the number of alter- 
natives offered for each question." 

This formula is especially convenient for use in correcting scores 
where the number of correct results are ahvays wanted. The fol- 
lowing formula is more convenient where the test is long and the 
number of errors only are considered as was done in the case of this 
study: 

True Score = n — _^ (w) 

c — 1 

where, 

n = the number of exercises in the test 

c =: the number of chances in each exercise of guessing the 
right answer 

w = the number of wrong answers 
To illustrate: Suppose a test contained 15 exercises. There were 
5 suggested answers in each exercise. A certain pupil made 12 errors 
on his paper. The answers given by the pupil were inconsistent 
with his training. It is required to find his probable true score. 

Substituting the numbers 15, 5, and 12 in the formula. 

True score = 15 —A (12) = 
4 

Instead of the pupil being credited with a score of 3 correct, 
a truer estimate of his real knowledge on this test is probably zero. 

To show how nicely this formula works out when the true 
status of conditions is known, the following additional illustration 
and analysis will make the point clearer. 

Suppose a test contains 20 exercises. In each exercise there are 
5 chances of selecting the correct answer. I know the answers in 
5 dififerent exercises for certain. I am ignorant entirely of the 
rest but take a chance at hitting the right ones. When my paper 
is returned to me 1 find that the examiner has credited me with 8 
answers correct. 

Now a score of 8 is absurd, for I knew only five answers. The 



54 Standardized Exercises in History 

extra three credits came to me by mere chance. A score of five is 
all I deserve; for, 

20 = the number of exercises in the test. 
5 = the number of answers really known. 
15 = the number of answers not known at all. 
3 = the number of answers possible to get correct by law 
of chance. 
5 -f" -^ ^^ ^ot^^ number of answers correct by knowledge and by 
chance. 
20 — 8 = 12, the number marked wrong by the examiner. 

True Score = 20 — —(12) — 5, which expresses exactly 
4 

the score I deserve. 



CHAPTER V 

CLASSIFICATION OF ALL THE STANDARDIZED EXER- 
CISES INTO SCALES C AND D AND TESTS 
E AND F 

Since the data gathered from Tests I, II, III, and Lists X and 
Y stood the proposed tests of reliability, all the 85 exercises were 
arranged in order of their value in terms of the sigma unit. This 
arrangement is given in Table XIV. 

A study of this table reveals the possibility of arranging the 
different exercises into scales and tests based upon their difficulty. 
Two scales and two tests are here proposed. Other arrangements 
are possible, but the ones given illustrate the theory of procedure. 
The organization of the material for Test E will be considered first. 

In constructing a standardized test the aim should be to include 
those points whose difficulty is as nearly equal as possible. By ref- 
erence to Table XIV it may be seen that that largest number of 
exercises which approximate the same value lie between the sigma 
values of 2.03 and 2.55. On each side of these points in the dis- 
tribution, the values do not run so closely together in such a large 
number of cases. These 32 exercises may be set aside tentatively as 
the possibility of a standardized test. 

On further examination of the remaining exercises the possi- 
bility of a scale seems evident. The word scale is here used in the 
sense of a series of exercises whose values differ equally from one 
another by some adopted unit. By moving the decimal point in 
the sigma values one place to the right, and then forcing the num- 
ber to the next unit if the first decimal figure is greater than .5 and 
conversely neglecting the decimal figure if equal to or less than .5, 
a series of 16 values may be found which differ from one another 
by two points. Such a series of exercises is offered in the form of 
Scale C. The actual sigma values and approximations are given in 
the score sheet for Scale C. 

On further examination of the distribution of the sigma values, 
the possibility of another scale is seen if a unit of .25 is taken as the 
basis. Such a scale is offered in the series of exercises in the form 
of Scale D. The values for the proposed scale are given in the 

55 



56 Standardized Exercises in History 

score sheet for Scale D. The guide to forming this scale was the 
succession of steps running from .50 up to 3.25. The 12 exercises 
presented approximate these steps quite closely. 

Table XIV 

THE ABSOLUTE VALUES OF ALL THE EXERCISES ARRANGED 

IN ORDER 

VALUE ABSTRACT OF CONTENT 

.43 The Mayflower was a ship. 

.54 The White men found the Indians here. 

.64 The Pilgrims came from England. 

.69 Sudden attacks from the Indians were among the hardships. 

.70 The Mayflower Compact was drawn up by the Pilgrims. 

.73 First houses were generally built of logs. 

.79 England once controlled all the colonies. 

.95 Men from Europe founded the colonies. 

1.03 Walking was the most common means of travel. 

1.04 The Mayflower crossed the Atlantic ocean. 

1.13 The Treaty Elm marked a lasting peace with the Indians in Pa. 

1.19 English was the most commonly spoken language. 

1.20 The colonies increased to 13 by the time of the Revolutionary War. 
\22 Pocahontas was an Indian girl. 

1.24 Pocahontas is said to have saved the life of John Smith. 

1.37 New York was founded by the Dutch. 

1.40 By the latter part of the 18th century democracy was wanted. 

1.41 The City of Brotherly Love was founded by William Penn. 

1.41 The "Great Treaty" was between the Indians and William Penn. 

1.46 The Charter Oak was a tree. 

1.50 The Dutch colony surrendered to the English. 

1.54 Candles furnished the light in the early homes. 

1.55 The capture of Quebec was a turning point in American history. 

1.58 The French and Indian War was the most important to the colonists. 

1.64 The English were the most numerous of all the nationalities. 

1.82 The Iroquois Indians assisted the English against the French. 

1.85 Corn was the principal native food crop. 

1.86 Tobacco was the main export crop from Va. 

1.87 The Quakers founded Pa. 

1.87 In the French and Indian War Washington gained experience for 
the Revolution. 

1.94 Emigration was generally westward. 

1.95 The Plymouth Rock is in Mass. 

2.03 Stuyvesant was the last of the series of Dutch governors. 

2.04 Massasoit kindly received the Pilgrims. 

2.04 The Act of Toleration was decreed by the Catholics in Md. 



Scales and Tests 57 

2.05 Miles Standish led in fighting the Indians. 

2.05 Jamesto\vn was founded in 1607. 

2.07 The French built a line of forts in western Pa. 

2.12 The Witchcraft Delusion occurred among the Puritans. 

2.13 The period of settlement was from 1600 on approximately. 
2.16 Roger Williams founded R. I. 

2.16 The first attempt of the English to colonize was in Va. 

2.23 Philadelphia was the seat of permanent political events. 

2.26 Slavery was introduced in 1619. 

2.28 The Quakers were persecuted by the Puritans. 

2.29 The Holy Experiment was tried out by William Penn. 

2.29 Geo. Oglethorpe founded Ga. to help poor debtors of England. 

2.31 Indentured servants held in Va. 

2.34 First settlement of Catholics in Colony of Md. 

2.37 First Pilgrims landed in 1620. 

2.37 First college founded was Harvard. 

2.38 Sir Edmund Andros was the Royal Governor of New En;^land. 
2.38 William Pitt the most outspoken friend in the English Parliament. 

2.38 Thomas Hooker led in founding Conn. 

2.39 Bacon's Rebellion was against Lord Berkeley. 
2.42 The Spanish once invaded Ga. 

2.44 The Patroons were Dutch landowners. 

2.45 New Sweden later became the colony of Delaware. 

2.46 Mason and Dixon line surveyed between Pa. and Md. 

2.47 Huguenots founded Charleston. 

2.50 Treaty of Paris established the English claims in America. 

2.51 The Germans settled mostly in Pa. 

2.53 The townmeeting settled local business affairs in New England. 

2.55 First settlement of the Swedes was in Delaware. 

2.61 Preachers the most influential professional class in New England. 

2.66 Ga. was the last colony founded. 

2.67 The last colony was founded in the 18th century. 

2.74 William Claibourne led a rebellion against Lord Baltimore. 

2.76 Braddock's Defeat occurred in the Fourth Inter-Colonial War. 

2.77 The founding of Pa. bridged the gap between the North and South. 
2.80 William Bradford was once leader of the Pilgrims. 

2.85 Conn, had the first written constitution springing from the people. 

2.92 John Winthrop was the leader in founding Boston. 

2.97 The Congregationalist Church grew out of the Puritan Church. 

3.01 The government of early Pa. was proprietary. 

3.02 The United Colonies of New England was formed in 1643. 

3.03 Rhode Island had the charter form of government. 

3.05 The right of woman's suffrage was granted for a time in New Jersev. 

3.15 Lord Delaw-are resided in England while governor of Va. 

3.32 Boston News Letter the first newspaper in America. 



S8 Standardized Exercises in History 

3.32 John Berkeley and Geo. Carteret once owned N. J. 

3.3 5 John Locke drew up the Grand Model. 

3.36 1619 marked the time of the first Representative Assembly in A. 

3.39 Gorges once proprietor of Maine. 

3.45 The Huguenots' settlement at New Rochelle was quite prosperous. 

Since Scale D is comprised of only 12 exercises its construction 
may be used to show how scales may be graphically illustrated. 
Figure 3 was drawn for this purpose and the explanation of it is as 
follows: Let the line A B represent the base line of the normal 
probability curve broken off at 2.5 on each side of the mean ordi- 
nate. This length is equivalent to a base line of 5 divisions with 

at one end and 5 at the other. If each of the sigma units be 
divided into 100 equal parts, the location of any sigma value is 
easily determined when the graph is drawn on cross section paper 
ruled 10 lines to the inch. 

Since the values of Scale D extend approximately from .50 
to 3.25 with a unit difference between any points of .25 , the 
location of the 12 exercises is readily apparent. In the absence of 
an exercise of zero and .25 difficulty the first exercise must be 
placed at .5 and the last at 3.25 . Exercises 3, 7, and 11 fall at 

1 (J. 2(5 and 3a respectively. The intervening values fall at the 
successive quarter points as illustrated in figure 3. All the values of 
the 85 exercises given in table XIV may be located on a base line 
(!i\i(led and subdivided as the line A B. In the case of Test E, 
described more fully below, all the 26 values fall within less than 
half the distance between two sigma points. 

In the formation of these two scales, values are needed extend- 
ing throughout nearly the whole range of the distribution of sigma 
values. Six out of the 32 exercises set aside tentatively to form a 
test were used in the formation of Scales C and D. The 26 remain- 
ing exercises out of the 32 set aside were used to make up what is 

fcr iic ^c 4^ 3? 

FIG. 3 SHOWING THE LOCATION OF EACH OF THE 12 
EXERCISES OF SCALE D. 



Scalt'S and Tests 59 

here presented as Test E. The values for each of the exercises in this 
test are given in its accompanying score sheet. By reference to it 
one sees the greatest possible difference between any two exercises 
is less than .50 sigma. When the values of the exercises are grouped 
into a series with a class interval of .05, the average sigma value 
of the series is found to be 2.29 ; the standard deviation from this 
central tendency is but .15, and the coefficient of dispersion based 
upon the average and standard deviation is only .068. These sta- 
tistical methods were applied to this series of exercises to test the 
homogeneity of the list. 

After Scales C and D and Test E had been formed, there 
remained 31 exercises whose values did not conform either to the 
definition of a true test or a true scale. These exercises are just as 
valuable for testing purposes though not so convenient to use. Their 
value for the diagnosis of a pupil's difficulties is even better, how- 
ever, than the other proposed measures since the range of the weight- 
ings is greater. These 31 exercises were then arranged in order 
of value with a score sheet and is here given as Test F. 

Test F is an illustration of a popular conception of the word 
test; that is, a series of questions or exercises whose value is 
simply known. In grading a pupil's work credit should be given 
on each point earned in proportion to its absolute value. Test E is 
an illustration of a true test wherein the value of all the questions 
or exercises is the same or approximately so. A pupil's score on 
such a test equals the sum of the points earned multiplied by the 
value common to each question or exercise. 

In a true scale the questions or exercises are arranged in order 
of the values, the easiest usually of zero difficulty being placed at 
the head of the list. As a pupil progresses in learning he is able to 
work farther down the list. His score is represented by the value 
of the last exercise worked successfully. Theoretically the effort 
required to progress from one point to the next is the same since 
the interval between any two points is a constant value. 

The words test and scale have been much used interchangeably. 
The distinction here given is offered to prevent a misconception of 
the two scales and two tests presented in this chapter. 



60 Stdiulnrdized Exercises in History 



SCALE C 

1. The Mayflower wns a chapel, hall, hold, plant, queen, ship. 

2. The Pilgrim Fathers came from England, France, Germany, Spain, 

Szveden. 

3. The "Thirteen Original Colonies" were at one time all under the politi- 

cal control of England, France, Holland, Spain, Sweden, 

4. The most common means of travel used by the earliest settlers was by 

aeroplane, railroad, steamboat, trolley-lines, <walking. 

5. Pocahontas is said to have saved the life of Sir William Berkeley, Jona- 

than Edwards, William Penn, John Rolfe, John Smith. 

6. The City of Brotherly Love was founded by ffilliam Berkeley, George 

Calvert, John Endicott, IVilliam Penn, John Smith, Miles Standish. 

7. Of the nationalities represented in all the colonies the most numerous 

were the Dutch, English, French, Germans, Irish, Scotch, Swedes. 

8. The principal native food crop was barley, corn, oats, potatoes, rice, 

rye, ivheat. 

9. The Toleration Act in Md. was decreed by the Catholics, Episcopalians, 

Puritans, Quakers. 

10. More lasting and prominent political events took place in Baltimore, 

Boston, Charleston, Ne^v York. Philadelphia than in any other one 
city. 

11. The Patroons were Dutch fishermen, fur traders, landowners, miners, 

preachers, teachers. 

12. The most influential professional class in New England were the editors, 

lawyers, physicians, preachers, teachers, writers. 

13. The first written constitution in the colonies springing from the people 

and creating a government was the colony of Conn., Del., Mass., 
N. J., Fa. 

14. The Charter form of government was enjoyed by the people of Del., 

Ga., Md., Pa., R. I., Fa. 

15. Lord Delaware lived in England nearly all the time he was governor 

of Conn., Del., N. J., R. I., Fa. 

16. The First Representative Assembly held in America was in 1&07, 1619, 

1620. 1643, 1754. 



Scales and Tests 



61 



SCORE SHEET FOR SCALE C 



No. of 


The 


Approximate 


Sigma 


Suggested 


Each 


Correct 


V^alue 


Value 


Practical 


Exercise 


Answer 






Value 


1 


Ship 


4 


.43 


1 


2 


England 


6 


.64 


1 


3 


England 


8 


.79 


1 


4 


Walking 


10 


1.03 


1 


5 


John Smith 


12 


1.24 


1 


6 


William Penn 


14 


1.41 


1 


7 


English 


16 


1.64 


2 


8 


Corn 


IS 


1.85 


2 


9 


Catholics 


20 


2.04 


2 


10 


Philadelphia 


22 


2.23 


2 


11 


Land Owners 


24 


2.44 


2 


12 


Preachers 


26 


2.61 


3 


13 


Connecticut 


28 


2.85 


3 


14 


Rhode Island 


30 


3.03 


3 


15 


\'irginia 


32 


3.15 


3 


16 


1619 


34 


3.36 


3 



62 Standardized Exercises in History 



SCALE D 

1. When the colonists first landed in America they found the natives to 

be Chinese, Indians, Malays, Negroes, IVIiite Men. 

2. The houses of the first colonists were generally built of brick, cement, 

dirt, logs, marble. 

3. Every one of the colonies was founded by men from Africa, Asia, Aus- 

tralia, Europe, South America. 

4. Pocahontas was an English, French, German, Indian, Spanish girl, 

5. One of the turning points in our Colonial History was marked by the 

capture of Acadia, Deerfield, Louisburg, Port Royal, Quebec in the 
French and Indian Wars. 

6. The Colonists and English soldiers were assisted in the French and 

Indian Wars by the loyalty of the Iroquois, Moliegan, Narragan- 
sett, Pequot, If'ampanoag Indian tribe. 

7. The last of the Dutch governors was Hudson, Minuit, Stuyvesant, IVin- 

throp, Yeardley. 

8. The Quakers in Mass. were persecuted by the Baptists, Dutch Re- 

formed, Methodists, Moravians, Puritans. 

9. The first settlement by the Swedes was in Conn., Del., Ga., Md., Mass., 

N. C, N. H., N. Y. 

10. William Bradford was once Leader of the Baptists, Catholics, Pilgrims, 

Quakers. 

11. The "United Colonies of New England" was formed in 1634, 1643, 

1683, 1733, 1774. 

12. The first newspaper in America was the Boston Neivs Letter, New York 

Times. Philadelphia Ledger, Poor Richard's Almanac, Providence 
Journal. 



Scales mid Tests 



63 



SCORE SHEET FOR SCALE D 



No. of 


The 


Approximate 


Sigma 


Suggested 


Each 


Correct 


Value 


Value 


Practical 


Exercise 


Answer 






Value 


1 


Indians 


.50 


.54 


1 


2 


Logs 


.75 


.73 


1 


3 


Europe 


1.00 


.95 


1 


4 


Indian 


1.25 


1.22 


1 


5 


Quebec 


1.50 


1.55 


2 


6 


Iroquois 


1.75 


1.82 


2 


7 


Stuyvesant 


2.00 


2.03 


2 


8 


Puritans 


2.25 


2.28 


2 


9 


Delaware 


2.50 


2.55 


3 


10 


Pilgrims 


2.75 


2.K0 


3 


11 


1643 


3.00 


3,02 


3 


12 


Boston News 
Letter 


3.25 


3.32 


3 



64 Standardized Exercises in History 

TEST E 

1. I'he Pil;:;iims were kindly received by Chief Cannnicns, Massasoii, 

Philip, PovjI/atan. 

2. Miles Standish led in fishtin^ the Dutdi, Indians, Puritans, Sivrdes. 

3. Jamestown was founded in 1607, 1619, 1620, 1643, 1733, 1754. 

4. To stop the movement of the Colonists westward the French soldiers 

built a line of forts in western Conn., Aid., Mass., Pa., N. J. 

5. The Witchcraft delusion occurred among the Baptists, Catholics, Dutch, 

Indians, Puritans. 

6. The period of settlement for most of the colonies took place in the cen- 

tury beginning with 1400, 1500, 1600, 1700, ISOO. 
1. Roger Williams founded the colony of Ga., Md., Mass., N. C. \. J., 

Pa., R. /.. S. C. Va. 
S. The first attempt of the English to colonize in America was in what 

was later called the colony of Conn., Del., Mass., Pa., Va. 
9. Slavery was first introduced in 1607, 1619, 1620, 1643, 1683, 1775, 1787. 

10. The "Holy Experiment" was tried out by irilliam Bradford, IFilliam 

Brcivstcr, Jacob Leisler, Peter Minuit, U'illiam Penn. 

11. To help the poor debtors of England James Oglethorpe founded the 

colony of Comi., Del., Ga., Md., N. J., S. C, la. 

12. Indentured servants and slaves were held mostly in Del., Ga,, Pa., 

S. C, Va. 

13. The first settlement of the Catholics was in Conn., Del., Md., Mass., 

N. C, N. I!., N. J., R. I. 

14. The first Pilgrims landed in 1607, 1619, 1620, 1643, 1683, 1776, 1789. 

15. The first college founded was Broivn, Harvard, Pennsylvania, Princeton, 

JVilliam and Mary, Yale. 

16. The most outspoken friend of the colonists in the English Parliament 

was Dinividdie, Montcalm, Pitt, Toianshend, JVolfe. 

17. As Royal Governor of New England Sir Edmond Andres represented 

the Common People, the Continental Conr/ress, the Ene/lish Croivn, 
the Proprietors. 

18. Thomas Hooker led emigrants from Mass. to found Conn., Del., Ga., 

Md., N. J., V. y. 

19. Nathaniel Bacon led a Rebellion against the tyranny of Governor 

Andros, Argall, Berkeley, Dale, Minuit, JVinthrop, Yeardley. 

20. The colony of Ga. was once invaded by the Dutch, French, Irish, 

Spanish, Sivedes. 

21. The New Sweden of 1638 later became known as the colony of Conn., 

Del.. .V. H., Pa., R. I. 

22. The Mason and Dixon Line was established between Pa. and Del., 

Md.. A'. J., A', y., Ohio. 

23. The colony of South Carolina was noted for the very early settlement 

of the Baptists, Catholics, Huguenots, Lutherans, Methodists in the 
founding of Charleston. 



Scales and Tests 



65 



24. The territorial claims of the English in America were established by 

the treaty of Aix-la-Chapelle, America, Paris, Rysivick, Utrecht. 

25. The Germans settled mostly in Md., N. J., N. Y., Pa., R. I., S. C, Va. 

26. Local business affairs in the New England settlements were generally 

managed by the governor, king, mayor, minister, toivn-meeting. 



SCORE SHEET FOR TEST E 



No. of 
Each Exercise 


The Correct 
Answer 


Approximate 
Value 


Sigma 
Value 


1 
2 
3 
4 
5 


Massasoit 

Indians 

1607 

Pennsylvania 

Puritans 


2 
2 
2 
2 
2 


2.04 
2.05 
2.05 
2.07 
2.12 


6 
7 
8 
9 
10 


1600 

Rhode Island 

Virginia 

1619 

William Penn 


2 
2 
2 
2 
2 


2.13 
2.16 
2.16 
2.26 
2.29 


11 
12 
13 
14 
15 


Georgia 

Virginia 

Maryland 

1620 

Harvard 


2 
2 
2 
2 
2 


2.29 
2.31 
2.34 
2.37 
2.37 


16 
17 
18 
19 
20 


Pitt 

English Crown 

Connecticut 

Berkeley 

Spanish 


I 

2 
2 

2 


2.38 
2.38 
2.38 
2.39 
2.42 


21 

22 
23 
24 
25 


Delaware 

Maryland 

Huguenots 

Paris 

Pennsylvania 


2 
2 
2 
2 
2 


2.45 
2.46 
2.47 
2.50 
2.51 


26 


Town-meeting 


2 


2.53 



66 Standardized Exercises in History 

TEST F 

1. Among the many hardships of the times were the unexpected attacks 

by Arabian, Chinese, Cossack, Indian, Negro warriors. 

2. The "Mayflower Compact" was drawn up by the Baptists, Huguenotf, 

Methodists, Pilgrims, Quakers. 

3. By the time the "Mayflower" had arrived at Plymouth Rock it had 

crossed the Antarctic, Arctic, Atlantic, Indian, Pacific ocean. 

4. Under the "Treaty Elm" William Penn made a lasting peace with the 

Canadians, Huguenots, Indians, Negroes, Puritans. 

5. The most commonly spoken language of early New England was Dutch, 

English, French, Scandinavian, Spanish. 

6. By the opening of the Revolutionary War the number of colonies had 

increased to tixelve, thirteen, fourteen, fifteen, sixteen. 

7. New York was founded by the Dutch, English, French, Spanish, Sivedes. 

8. By the latter part of the 18th century the form of government most 

desired by the colonists was aristocracy, autocracy, democracy, 
monarchy, oligarchy. 

9. The Great Treaty was between the Indians and Governor Berkeley, 

John Endicott, Cotton Mather, IVilliam Penn, John Smith, Miles 
Standish. 

10. The Charter Oak was a church, meeting house, painting, school, toivn 

hall, tree. 

11. The political control of the Dutch in America came to an end when 

Governor Stuyvesant surrendered to the English, French, Germans, 
Spaniards, Sivedes. 

12. The light at night in these early homes came from candles, electricity, 

gas, kerosene, phosphorus. 

13. The most important of the Four Inter-Colonial Wars to the colonists 

was Queen Anne's, King George's, French and Indian, King 
William's. 

14. The main export crop of early Va. was corn, cotton, indigo, rice, tobacco, 

luheat. 

15. The Quakers founded the colony of Ga., Mass., N. H., Pa., S. C. 

16. George Washington was fitted for leadership in the Revolutionary War 

through his experiences in the Colonial War of Queen Anne, King 
George, French and Indian, King William. 

17. In emigrating the colonists generally went east, north, south, ivest. 

18. The "Plymouth Rock" is in Conn., Del., Md., Mass., N. J., N. Y., Pa., 

R. I., S. C. 

19. The last of the thirteen colonies founded was Conn., Del., Ga., Md., 

Mass., N. C, N. Y., Pa., S. C, R. I., Fa. 

20. The last colony was founded in the ISth, 16th, 17th, 18th, 19th century. 

21. William Claiborne led a Rebellion against Lord Baltimore, Lord Berkeley, 

Lord Brook, Lord Delavjare, Lord Say, 



Scales and Tests 67 



22. ''Braddock's Defeat" occurred in the first, second, tliird, fourth, Colo- 

nial War. 

23. The gap in location, age, and ideals between the Northern and South- 

ern colonies was bridged over by the founding of Del., Md., N. J., 
N. Y., Pa. 

24. The leadership of John Winthrop was noted in the founding of 

Baltimore, Boston, Charleston, Philadelphia, Providence. 

25. The strict Puritan Church of early New England gradually gave way 

to what has since become known as the Baptist, Congregationalist, 
Lutheran, Methodist, Spiritualist church. 

26. The Proprietary form of government was exercised in Mass., N, Y., 

Pa., R. I., S. C, Fa. 

27. The right of woman suffrage was granted for a time in Conn., Del., 

Mass., N. J., Va. 

28. John Berkley and George Cartaret once owned Conn., Del., Ga., Md., 

Mass., N. H., N. J. 

29. The Grand Model was drawn up by Cecil Calvert, King Charles, 

John Locke, John Smith. 

30. The colony of Maine was once under the proprietorship of John 

Davenport, Sir Ferdinando Gorges, Anne Hutchinson, fVilliam 
Kieft, Roger IVilliams. 

31. The welfare of the New York colony was much helped by the very 

early settlement of the Huguenots at Albany, New Burg, Neiu 
Rochelle, Saratoga, West Point. 



68 



Standardized Exercises in History 



SCORE SHEET FOR TEST F 



No. of _ 
Each Exe cise 


The Correct 
Answer 


Approximate 
Value 


Sigma 
Value 


1 
2 
3 
4 
5 


Indians 
Pilgrims 
Atlantic 
Indians 
English 




.69 

.70 

1.04 

1.13 

1.19 


6 
7 
8 
9 
10 


Thirteen 
Dutch 
Democracy 
William Penn 
Tree 




1.20 
1.37 
1.40 
1.41 
1.46 


11 
12 
13 
14 
15 


English 

Candles 

French and Indian 

Tobacco 

Pennsylvania 




1.50 
1.54 

1.58 
1.86 
1.87 


16 
17 
18 
19 
20 


French and Indian 

West 

Massachusetts 

Georgia 

18th Century 


2 
3 
3 


1.87 
1.94 
1.95 
2.66 
2.67 


21 
22 
23 
24 
25 


Lord Baltimore 

Fourth 

Pennsylvania 

Boston 

Congregationalist 


3 
3 
3 
3 
3 


2.74 
2.76 
2.77 
2.92 
2.97 


26 
27 
28 
29 
30 


Pennsylvania 
New Jersey 
New Jersey 
John Locke 
Gorges 


3 
3 
3 
3 
3 


3.01 
3.05 

3.32 
3.35 
3.39 


31 


New Rochelle 


3 


3.45 



CHAPTER VI 

USE OF THE STANDARDIZED EXERCISES FOR 
SUPERVISORY PURPOSES 

Sec, 1. — Reorganization of Proposed Measures into Test A 
mid Test B. Part I and Part II. The formation of the Scales C and 
D and Tests E and F illustrate the theoretical procedure in organizinj^ 
standardized testing material for supervisory purposes. In the opin- 
ion of the author they are reliable and may be used by administra- 
tive officers in the form given. However, since these scales and 
tests were designed from the beginning to be of the greatest prac- 
tical value possible, it was deemed advisable to recast all the fore- 
going measures into two tests. It was thought that such a revision 
would render the use of the standardized exercises much easier for 
class room teachers. One of the reasons for the wide and accepted 
use of the Courtis Arithmetic Tests, the Monroe Reading Test, 
and the Ayres Handwriting Scale is that these measures are so easily 
understood by those just beginning the measurement of class room 
products. A number of other educational measures, all theoretically 
correct, are not in use simply because they are too difficult to under- 
stand and use by those who should use them most. The writer 
therefore has sought to meet the needs of those who want to begin 
using standardized testing material by recasting all the foregoing 
Scales and Tests into the forms of Test A, and Test B, Part I 
and Part II. 

The basis of this reclassification was to put into Test A all 
those exercises whose sigma value was 2 or approximately 2. This 
approximation includes all the exercises of Test E; exercises 13-18 
of Test F; exercises 7-11 in Scale C; and exercises 5-8 in Scale D. 
Test B was made up into Part I and Part II. Into Part I were 
put all the remaining easy exercises with a value approximating 1, 
and into Part II all the remaining difficult exercises whose value 
approximates a value of 3. By so doing nothing is lost in reliability 
and only eight per cent in accuracy. So far as the author can learn 
from supervisors who have tried the tests the approximations are 
valid and readily used by class room teachers. 

Test A, and Test B with manual of "Directions for Adminis- 

69 



70 Standardized Exercises in History 

tering, Scoring and Using the Results" are here reproduced for the 
convenience of those who wish to use the tests. Judging by the 
numerous inquiries for the Tests, there seems to be a real profes- 
sional interest in such measures. They are not offered as finished 
products but as the beginning of a series of tests which should be 
more accurately gauged because of the experience offered in the 
derivation of these tests. While either test may be used separately, 
both should be used if the achievement of a class is to be carefully 
determined. The content of the 85 exercises in the two tests in- 
clude all the points usually covered in the elementary school. 



Test A Score 

UNIVERSITY OF PITTSBURGH 
School of Education 

EXERCISES IN UNITED STATES HISTORY 
Colonial Period 
sturgiss b. davis 

Directions to Pupils — Fill in these blank lines with: 

Your name School 

Grade ; Age ; City ; Date 



On the succeeding pages are historical exercises like the following: 

1. Columbus discovered Africa, America, Asia, Australia, Europe. 

There are several suggested answers, only one of which is correct. 
That one of course is America. Draw a heavy line clear around it. 
This exercise is now worked out and may be read as a correct historical 
statement, Columbus discovered America. 

2. The New World was discovered in the year WOO, U53, 1492, 1497, 

1565. 

Draw a line around the date 1492. The correct historical state- 
ment is, The New World was discovered in the year 1492. 

3. The first man to sail around the world was Balboa, Cortez, DeSoto, 

Drake, Magellan. 

Draw a line around the word Magellan. What is the correct his- 
torical statement on which this exercise is based? 

4. Try to work out each exercise whether you are sure of the answer or 
not. Select the one which you think is right and draw a line around 
it. There is alzuays one correct answer in each exercise. 

5. The time limit is 12 minutes. This is plenty of time if you work stead- 
ily, and do not look around. 

6. Your teacher will give you all other needed directions. Listen carefully. 



Use for Supervisory Purposes 71 

TEST A 

1. The Pilgrims were kindly received by Chief Canonicus, Massasoit, 

Philip, Poiv/iatan. 

2. Miles Standish led in fighting the Dutcli, Indians, Puritans, Sivrdes. 

3. Jamestown was founded in 1607, 1619, 1620, 1643, 1733, 1754. 

4. To stop the movement of the Colonists westward the French soldiers 

built a line of forts in western Conn., Mass., Md., Pa., N. J. 

5. The Witchcraft delusion occurred among the Baptists, Catholics, Dutch, 

Indians, Puritans. 

6. The period of settlement for most of the colonies took place in the cen- 

tury beginning with 1400, 1500, 1600, 1700, 1800. 

7. Roger Williams founded the colony of Ga., Md., Mass., N. C, N. J., 

Pa., R. I., S. C, Fa. 

8. The first attempt of the English to colonize in America was in what 

was later called the colony of Conn., Del., Mass., Pa., Fa. 

9. Slavery was first introduced in 1607, 1619, 1620, 1643, 1683, 1775, 1787. 

10. The "Holy Experiment" was tried out by William Bradford, William 

Breijjster, Jacob Leisler, Peter Minuit, William Penn. 

11. To help the poor debtors of England James Oglethorpe founded the 

colony of Conn., Del., Ga., Md., N. J., S. C, Fa. 

12. Indentured servants and slaves were held mostly in Del., Ga., Pa., 

S. C, Fa. 

13. The first settlement of the Catholics was in Conn., Dei, Md., Mass., 

N. C, N. H., N. J., R. I. 

14. The first Pilgrims landed in 1607, 1619, 1620, 1643, 1683, 1776, 1789. 

15. The first college founded was Broivn, Harvard, Pennsylvania, Princeton, 

William and Mary, Yale. 

16. The most outspoken friend of the colonists in the English Parliament 

was Dinividdie, Montcalm, Pitt, Wolfe, Toiunshend. 

17. As Royal Governor of New England Sir Edmond Andros represented 

the Continental Congress, the English Crown, the Proprietors, the 
Common People. 

18. Thomas Hooker led emigrants from Mass. to found Conn., Del., Ga., 

Md., N. J., N. Y. 

19. Nathaniel Bacon led a Rebellion against the tyranny of Governor Argall, 

Andros, Berkeley, Dale, Minuit, Winthrop, Yeardley. 

20. The colony of Ga. was once invaded by the Dutch, French, Irish, 

Spaniards, Sivedes. 

21. The New Sweden of 1638 later became known as the colony of Conn., 

Del., N. H., Pa., R. I. 

22. The Mason and Dixon line was established between Pa. and Del., 

Md., N. J., N. Y., Ohio. 

23. The colony of South Carolina was noted for the very early settlement 

of the Baptists. Catholics, Huguenots, Lutherans, Methodists in the 
founding of Charleston. 



72 Standardized Exercises in History 

24. The territorial claims of the English in America were established by 

the treaty of Aix-la-Chapelle, America, Paris, Rysivick, Utrecht. 

25. The Germans settled mostly in Md., N. J., N. Y., Pa., R. I., S. C, Fa. 

26. Local business affairs in the New England settlements were generally 

managed by the governor, king, mayor, minister, toivn-meeting. 

27. The most important of the Four Inter-Colonial Wars to the colonists 

was Queen Anne's, King George's, French and Indian, King 
IVilliam's. 

28. The main export crop of early Va. was corn, cotton, indigo, rice, tobacco, 

lulieat. 

29. The Quakers founded the colony of Ga., Mass., N. H., Pa., S. C. 

30. George Washington was fitted for leadership in the Revolutionary War 

through his experiences in the Colonial War of Queen Anne, King 
George, French and Indian, King Wilitam. 

31. In emigrating the colonists generally went east, north, south, west. 

32. The "Plymouth Rock" is in Conn., Del., Md., Mass., N. J., N. Y., Pa., 

R. I., S. C. 

33. One of the turning points in our Colonial History was marked by the 

capture of Arcadia, Deerfield, Louisburg, Port Royal, Quebec, in the 
French and Indian Wars. 

34. The Colonists and English soldiers were assisted in the French and 

Indian Wars by the loyalty of the Iroquois, Mohegan, Narragansett, 
Pequot, Wampanoag Indian tribe. 

35. The last of the Dutch governors was Hudson, Minuit, Stuyvesant, 

Winthrop, Yeardley. 

36. The Quakers in Mass. were persecuted by the Baptists, Methodists, 

Moravians, Puritans, Dutch Reformed. 
111. Of the nationaPities represented in all the colonies the most numerous 
were the Dutch, English, French, Germans, Irish, Scotch, Swedes. 

38. The principal native food was barley, corn, oats, potatoes, rice, rye, 

wheat. 

39. The Toleration Act in Md. was decreed by the Episcopalians, Catholics, 

Puritans, Quakers. 

40. More lasting and prominent political events took place in Baltimore 

Boston, Charleston, New York, Philadelphia than in any other one 
city. 

41. The Patroons were Dutch fishermen, fur traders, landowners, miners, 

preachers, teachers. 

Number Right X 2 = Score 

Test A Score 



Use for Supervisory Purposes 73 

UNIVERSITY OF PITTSBURGH 
School of Education 

EXERCISES IN UNITED STATES HISTORY 
Colonial Period 
sturgiss b. davis 

Directions to Pupils — Fill in these blank lines with: 

Your name School 

Grade ; Age ; City ; Date 

On the succeeding pages are historical exercises like the following: 

1. Columbus discovered Africa, America, Asia, Australia, Europe. 

There are several suggested answers, only one of which is correct. 
That one of course is America. Draw a heavy line clear around it. 
This exercise is now worked out and may be read as a correct historical 
statement, Columbus discovered America. 

2. The New World was discovered in the year 1000, 1453, 1492, 1497, 

1565. 

Draw a line around the date 1492. The correct historical state- 
ment is, The New World was discovered in the year 1492. 

3. The first man to sail around the world was Balhoa, Cortez, DeSoto, 

Drake, Magellan. 

Draw a line around the word Magellan. What is the correct his- 
torical statement on which this exercise is based? 

4. Try to work out each exercise whether you are sure of the answer or 
not. Select the one which you think is right and draw a line around 
it. There is alivays one correct answer in each exercise. 

5. The time limit is 12 minutes. This is plenty of time if you work stead- 
ily, and do not look around. 

6. Your teacher will give you all other needed directions. Listen carefully. 

TEST B— PART I 

1. Among the many hardships of the time were unexpected attacks by 

Arabian, Indian, Chinese, Cossack, Negro warriors. 

2. The "Mayflower Compact" was drawn up by the Baptists, Huguenots, 

Methodists, Pilgrims, Quakers. 

3. By the time the "Mayflower" had arrived at Plymouth Rock it had 

crossed the Atlantic, Antarctic, Arctic, Indian, Pacific ocean. 

4. Under the "Treaty Elm" William Penn made a lasting peace with the 

Canadians, Huguenots, Indians, Negroes, Puritans. 

5. The most commonly spoken language of early New England was Dutch, 

English, French, Scandinavian, Spanish. 

6. By the opening of the Revolutionary War the number of colonies had 

increased to iiuelve, thirteen, fourteen, fifteen, sixteen. 



74 Standardized Exercises in History 

7. New York was founded by the Dutch, English, French, Spanish, Swedes. 

8. By the latter part of the 18th century the form of government most 

desired by the colonists was aristocracy, autocracy, democracy, 
monarchy, oligarchy. 

9. The Great Treaty was between the Indians and Governor Berkeley, 

John Endicolt, Cotton Mather, JVilliam Penn, Jolin Smith, Miles 
Siandish. 

10. The Charter Oak was a church, meeting house, painting, school, toivn 

hall, tree. 

11. The political control of the Dutch in America came to an end when 

Governor Stuyvesant surrendered to the English, French, Germans, 
Spaniards, Sivedes. 

12. The light at night in these early homes came from candles, electricity, 

gas, kerosene, phosphorus. 

13. When the colonists first landed in America they found the natives to 

be Chinese, Indians, Malays, Negroes, JVhite-men. 

14. The houses of the first colonists were generally built of bricks, cement, 

dirt, logs, marble. 

15. Every one of the colonies was founded by men from Africa, Asia, 

Australia, Europe, South America. 

16. Pocahontas was an English, French, German, Indian, Spanish girl. 

17. The Mayflower was a chapel, hall, hotel, plant, queen, ship. 

18. The "Pilgrim Fathers" came from England, France, Germany, Spain, 

Siveden. 

19. The "Thirteen Original Colonies" were at one time all under the 

political control of England, France, Holland, Spain, Siveden. 

20. The most common means of travel by the earliest settlers was by 

aeroplane, railroad, steamboat, trolley lines, nvalking. 

21. Pocahontas is said to have saved the life of Sir IVilliam Berkeley, 

Jonathan Edwards, IVilliam Penn, John Rolfe, John Smith. 

22. The City of Brotherly Love was founded by IVilliam Berkeley, George 

Calvert, John Endicott, JVilliam Penn, John Smith, Mites Standish. 

Number Right X 1 = Score 

TEST B— PART II 

23. The last of the thirteen colonies founded was Conn., Del., Ga., Md., 

Mass., N. C, N. Y., Pa., S. C, R. I., Va. 

24. The last colony was founded in the 1 5th, 16th 17th, 18th, 19th century. 

25. William Claiborne led a Rebellion against Lord Berkeley, Lord Brook, 

Lord Baltimore, Lord Delaware, Lord Say. 

26. "Braddock's Defeat" occurred, in the First, Second, Third, Fourth of 

the French and Indian, or Inter-Colonial Wars. 

27. The gap in location, age, and ideals between the Northern and South- 

ern colonies was bridged over by the founding of Del., Md., N. J., 
N. Y., Pa. 



Lhe for Supervisory Purposes 75 

28. The leadership of John Winthrop was noted in the founding of 

Baltimore, Boston, Charleston, Philadelphia, Providence. 

29. The strict Puritan Church of early New England gradually gave way 

to what has since become known as the Baptist, Congregationaltst, 
Lutheran, Methodist, Spiritualist church. 

30. The Proprietary form of government was exercised in Mass., N. Y., Pa., 

R. I., S. C, Fa. 

31. The right of woman suffrage was granted for a time in Conn., Del., 

Mass., N. J., Fa. 

32. John Berkley and George Cartaret once owned Conn., Del., Ga., Md., 

Mass., N. H., N. J. 

33. The Grand Model was drawn up by Cecil Calvert, King Charles, John 

Locke, John Smith. 

34. The colony of Maine was once under the proprietorship of John 

Davenport, Sir Ferdinajido Georges, Anne Hutchinson, JFilliam 
Kieft, Roger IVilliams. 

35. The welfare of the New York colony was much helped by the very 

early settlement of the Huguenots at Albany, Nevu Burg, New 
Rochelle, Saratoga, JVest Point. 

36. The first settlement by the Swedes was in Conn., Del., Ga.. Md., Mass., 

N. C, N. H., N. Y. 

37. William Bradford was once Leader of the Baptists, Catholics, Pilgrims, 

Quakers. 

38. The "United Colonies of New England" was formed in 1634, i6f^3, 

1683, 1733, 1774. 

39. The first Newspaper in America was the Boston Nevjs Letter, Nevj York 

Times, Philadelphia Ledger, Providence Journal, Poor Richard's 
Almanac. 

40. The most influential professional class in New England were the editors, 

lavjyers, physicians, preachers, teachers, vnriters. 

41. The first written constitution In the colonies springing from the people 

and creating a government was in the colony of Conn., Del., Mass., 
N. J., Fa. 

42. The Charter form of government was enjoyed by the people of Del., 

Ga., Md., Pa., R. I., Fa. 

43. Lord Delaware lived in England nearly all the time he was governor 

of Conn., Del., N. J., R. I., Fa. 

44. The First Representative Assembly held in America was in 1607, 1619, 

1620, 1643, 1754. 

Number Right X 3 = Score 



76 Standardized Exercises in History 

^Directions for Administering, Scoring, Using the Results 

DIRECTIONS FOR ADMINISTERING EXERCISES IN 
UNITED STATES HISTORY 
To Teachers: 

1. These exercises are not intended to be used for testing purposes until 
the pupils have been formally instructed in the period of history on which 
the test is based. 

2. Endeavor in so far as possible to preserve the normal attitude of 
mind, both for yourself and your pupils. Let the exercises be presented as 
an integral part of the usual school procedure. 

3. Any special coaching, or announcement of the test previous to the 
hour of giving it is entirely inconsistent with good work in educational 
measurement. 

4. At the time you plan to administer the test ask the pupils to clear 
their desks and see that their pencils are sharp. 

5. Say to them: "Today we are going to find out how well we learned 
many of the points we studied in our history lessons. I shall give to each 
of you a folder in which are some exercises to be worked out. You may 
look at the first page, but no more, until I give the signal to start working." 

6. Distribute the tests by placing on the first desk of each row suffi- 
cient copies to supply the pupils of the row. 

7. When each pupil has a copy, direct everyone to fill in the blank 
spaces at the top of the first page. 

8. After the blanks have been filled out, say to them: ''You may now 
look at the first illustrative exercise on the front page. Read it over silently 
as I read it aloud." 

'1. Columbus discovered Africa, America, Asia, Australia, 
Europe.' 

"There are several suggested answers, only one of which is cor- 
rect. That one is, of course, America. Draw a heavy line clear 
around it. This exercise is now worked out and may be read as a 
correct historical statement, Columbus discovered America." 

9. "Look at the next exercise. It reads, 'The New World was discov- 
ered in the year 1000, 1453, 1492, 1497, 1565.' Draw a line around the date 
1492. The correct historical statement is. The New World was discovered in 
the year 1492." 

10. "Now look at the third illustration, 'The first man to sail around 
the world was Balboa, Cortez, DeSoto, Drake, Magellan.' Draw a line 

*Reproduced from Manual to Teachers, title page omitted. 



Use for Supervisory Purposes 77 

around the word Magellan. What is the correct historical statement on 
which this exercise is based?" Let some pupil read it. 

11. "All the other exercises in this test are worked out in the same 
way, that is, by drawing a line around the right one of the several sug- 
gested answers." 

12. "Try to work out each exercise whether you are sure of the right 
answer or not. Select the one which you think is right and draw a line 
around it. There is always one correct answer in each exercise." 

13. "The time limit is given on the test. This is plenty of time if 
you work steadily and do not look around." 

14. "Does every one understand clearly just what we are going to 
do?" 

15. Take the time and trouble to answer any questions relating to 
the directions for working, if there be such. 

17. At the end of the time limit say, "STOP! I shall now read the 
right answers and you may correct your own papers. Place an X before 
the number of each exercise you missed. Do not mark the correct ones." 

18. Read the exercises as correct historical statements slowly and dis- 
tinctly. No erasures are permitted. 

19. At this point pupils often ask very good questions. It is a psy- 
chological moment to drive home facts that have been forgotten. 

20. When the pupils are satisfied that all the errors have been marked, 
direct that the papers be passed forward, and collect them. 

SCORING 

1. Each exercise is counted as wholly correct or incorrect. 

2. An exercise not tried is counted as wrong. 

3. Each correct exercise has a value of: 

2 in Test A 

1 in Test B; Part I 

3 in Test B; Part II 

4. A pupil's score is the sum of the values of the exercises worked 
correctly. Note the formula at the bottom of each list of exercises. 

5. The purpose of letting the pupils mark their own papers is to call 
attention to their own errors. These marked errors also assist the teacher 
in scoring. If the class is a reliable one only those papers need to be re- 
viewed carefully which show an unusually high or low number of errors. 
Or such a class may score their own papers. 

USING THE RESULTS 
1. The median, or most representative score for the class, may be 



78 Standardized Exercises in History 

quickly found by arranging the papers in order of their scores, and select- 
ing the middle paper. Such medians may be used for comparative purposes. 

2. Such comparisons are of little constructive value. Class scores and 
pupils' scores are only indications of the trend of progress. A larger use 
for the scores obtained is to find out what specific suggestions they give for 
the improvement of instruction. These Tests are especially adapted for this 
purpose thru a study of the errors made by pupils on any given exercise. 
For the purpose of illustration, suppose that in scoring the papers it was 
noted that many pupils failed on such an exercise as, "The first man to sail 
around the world was Balboa, Cortes, DeSoto, Drake, Magellan." List all 
the suggested answers in a column thus, 

Balboa, Then taking each paper separately tabulate the 

Cortez, number of times each answer was selected. If the 

DeSoto, tabulation is fairly well distributed* among the 

Drake, five, it indicates that most of the pupils have prob- 

Magellan. ?b!y no definite idea of this point; if on the other 

hand the majority of mistakes are centered at the word "Drake"t it suggests 

that many pupils have not cleared up the distinction between the voyages 

of Magellan and Drake. These are only two of several other possibilities 

such a study of errors will reveal. But under either condition it is evident 

that this point needs to be given renewed attention in class. 

Having found this specific source of low scores the cause can usually 
be found; and the proper class procedure to prevent its repetition is gen- 
erally open to teachers familiar with the laws of learning. 

3. It should be noted that all the suggested answers in the exercises 
are conservative historical material, and may be used for rapid review work. 
Some are more important than others but all have an impressionistic educa- 
tional value. None of the points are worthy material in themselves as 
such ; but as material for constructive thinking on the historical problems 
of this period, they are worth while. 

*Suppose the class consisted of 25 pupils and the distribution was as in 
Illustration I. 

^Illustration II shows how the answers would be distributed. 

I.— Balboa 11111 1 

Cortez, 111 . „ , 

DeSoto, 11111 
Drake, 11111 
Magellan, 111111 

II.— Balboa, 
Cortez, 
DeSoto, 

Drake, 11111 11111 1 
Magellan, 11111 11111 1111 



Use for Supervisory Purposes 79 

SCORE SHEET— TEST A 

No. of each exercise Correct Answer 



1 


MASSASOIT 


2 


INDIANS 


3 


1607 


4 


PENNSYLVANIA 


5 


PURITANS 


6 


1600 


7 


RHODE ISLAND 


8 


VIRGINIA 


9 


1619 


10 


WILLIAM PENN 


11 


GEORGIA 


12 


VIRGINIA 


13 


MARYLAND 


14 


1620 


15 


HARVARD 


16 


PITT 


17 


ENGLISH CROWN 


18 


CONNECTICUT 


19 


BERKELEY 


20 


SPANISH 


21 


DELAWARE 


22 


MARYLAND 


23 


HUGUENOTS 


24 


PARIS 


25 


PENNSYLVANIA 


26 


TOWN-MEETING 


27 


FRENCH AND INDIAN 


28 


TOBACCO 


29 


PENNSYLVANIA 


30 


FRENCH AND INDIAN 


31 


WEST 


32 


MASSACHUSETTS 


33 


QUEBEC 


34 


IROQUOIS 


35 


STUYVESANT 


36 


PURITANS 


37 


ENGLISH 


38 


CORN 


39 


CATHOLICS 


40 


PHILADELPHIA 


41 


LANDOWNERS 



80 Staiidardizcd Exercises i?i Ilhtorx 







SCORE SHEET— TEST B, PART I 


No. 


of each 


exercise Correct Answer 




1 


INDIANS 




2 


PILGRIMS 




3 


ATLANTIC 




4 


INDIANS 




5 


ENGLISH 




6 


THIRTEEN 




7 


DUTCH 




8 


DEMOCRACY 




9 


WILLIAM PENN 




10 


TREE 




11 


ENGLISH 




12 


CANDLES 




13 


INDIANS 




14 


LOGS 




15 


EUROPE 




16 


INDIAN 




17 


SHIP 




18 


ENGLAND 




19 


ENGLAND 




20 


WALKING 




21 


JOHN SMITH 




22 


WILLIAM PENN 
SCORE SHEET— TEST B, PART II 


No, 


of each 


exercise Correct Answer 




23 


GEORGIA 




24 


18TH CENTURY 




25 


LORD BALTIMORE 




26 


FOURTH 




27 


PENNSYLVANIA 




28 


BOSTON 




29 


CONGREGATIONALIST 




30 


PENNSYLVANIA 




31 


NEW JERSEY 




32 


NEW JERSEY 




33 


JOHN LOCKE 




34 


GORGES 




35 


NEW ROCHELLE 




36 


DELAWARE 




37 


PILGRIMS 




38 


1643 




39 


BOSTON NEWS LETTER 




40 


PREACHERS 



Use for Sii/^erzi'isory Purposes 81 



41 CONNECTICUT 

42 RHODE ISLAND 

43 VIRGINIA 

44 1619 



Sec. 2. — Vnliie of the Standardized Exercises. After standard- 
ized testing material is organized into as easy a form as possible, it 
is pertinent to inquire into its educational value. Supposing that a 
class is tested and the scores tabulated, what really is shown? More 
specifically, when such measures as Test A and Test B are given to 
pupils, what is measured ? 

The answer to this question has been volunteered without men- 
tion of it by several teachers who have just become i:iterested in 
educational measures. Immediately the idea is ofiered that these 
particular tests reveal pupils' ability in verbal memory of historical 
facts and in this ability alone. In other words, the responses are 
mechanical. Such an evaluation of these tests can not be supported 
either by a logical analysis of the exercises or case studies of pupils 
at v.'ork on the tests. 

When an analysis of an exercise is made, it shows that much 
more than verbal memory may be involved in working out the cor- 
rect answers. Just because a pupil does not write out a long para- 
graph in the selection of the correct one of the several suggested 
answers, does not justify the position that verbal memory only was 
employed in arriving at a decision. In order to respond with such 
a simple answer as yes or no to a given query may involve associa- 
tive memory, logical selection, imagination, and judgment of the most 
complex nature. As the writer has watched pupils work, and lis- 
tened to their thinking out loud in experiments, he is convinced that 
thinking of a very complex type has often been involved. Take such 
an exercise as, "The Patroons were Dutch fishermen, fur traders, 
landowners, miners, preachers, teachers." It was quite possible for 
these early settlers to have been engaged in any one of these activi- 
ties except possibly mining. For the pupil whose knowledge on this 
point is not clear a whole complex of associations and selective judg- 
ments may be necessary to arrive at what seems to him to be the 
logical choice. Since there can be but one accepted answer all the 
other suggestions must be eliminated. In the eliminating process 
each answer must be taken up separately and considered in the light 



82 Standardized Exercises in History 

of known facts. They might have been fishermen for they settled 
where water was in abundance, but no facts are known about the 
manufacture of fish products, or the sale of them direct. Miners 
would not be a good choice for nothing is known of this region as 
producing minerals of value ; preachers is certainly not the correct 
one for nothing is known about the need for a great number of 
them in a settlement where religion was not the foremost ques- 
tion ; landowners is the only suggested answer that brings with it 
any associations whatever that appear sound and so it becomes the 
logical choice of the lot. Just because all these mental processes 
take place almost instantly teachers are apt to conclude that the 
mental process was verbal memory. But the situation is no different 
from that of the business man who decides momentous financial 
questions seemingly on the spur of the moment. However, any one 
familiar with the conditions knows that he has gone through a whole 
train of thinking before giving his answer. Pupils do just as quick 
thinking on the playground. Of course it must be conceded that 
it is just possible that the opposite might occur but it would result 
from artificial conditions. Pupils are not taught history by the "1001 
question and answer" method in schools of the present. The mate- 
rial is well organized and logically presented. Pupils who do the 
work of the school respond much as they are instructed. 

The analysis just given above is not theoretical but illustrative 
of the kind of thinking which the pupils did whom the writer exam- 
ined. In general it may be said that when these pupils knew the 
right answer they expressed it at once, but when they were in doubt 
they went through a process of logical selection and eliminated one 
possibility after the other. This was especially true of eighth grade 
pupils who brought to bear all their historical knowledge about other 
periods of history, and their knowledge of geography. One typical 
illustration taken verbatim from a pupil's "thinking out loud" must 
suffice. The exercise in question was: "The Mason and Dixon 
Line was established between Pa. and Del., Md., N. J., N. Y., 
Ohio." "In the south somewhere; not N. Y. ; it is east; line runs 
from east to west ; Ohio would make it north and south ; it must be 
between Pa. and Md." Not all the pupils were so fortunate in 
their process of elimination but it seems clear that there was always 
some academic basis for the answer chosen. 



Use for Supervisory Purposes 83 

In further reply to a prior criticism made of the value of this 
form of testing a comparison of the scores earned by 245 pupils in 
the Pressey Rote Memory Test and Test III of the History Exer- 
cises is offered as more evidence that the criticism cannot be sup- 
ported. The comparison was made on the assumption that if rote 
memory was largely involved the same pupils ought to make high 
scores or low scores consistently throughout in both tests. In other 
words the scores of pupils in both tests ought to show a high degree 
of positive correlation. When the scores from each test are tabulated 
in a correlation table the results are shown in Table XV. An 
examination of the frequencies at the right and bottom of the table 
shows how differently pupils reacted in each instance. With but 
one exception the scores from the memory test range throughout 
the 20 possibilities ; the range of scores in the history test however 
is but 12. While the distribution of the frequencies in each instance 
approximates the probability curve the effects of other mental proc- 
esses at work are clearly evident on the side of history. The 
absence of correlation is further revealed in the distribution of the 
scores on the surface of the table. Instead of the scores as a whole 
running parallel with the diagonal of high positive correlation the 
tendency is just the opposite. In fact the coefiicient of relationship 
is negligible, being only — .06. The outcome of this experiment 
is a more exact basis for the claim that the history exercises here 
presented test a high quality of thinking when pupils work them out. 

In specific answer to the question raised at the beginning of 
this section as to what Tests A and B really measure, it should be 
stated again that the original purpose of the tests was to measure 
only the amount of historical information pupils possess relative to 
the content of the exercises. How pupils gained their information; 
what abilities they exercised in working out the tests ; why they 
succeeded or failed — none of these questions entered into the pri- 
mary purpose of the study. The value of the tests must be judged 
by the results of their use in actual school conditions. Evidence 
has been offered to show that the information which pupils produced 
was not a mechanical process alone as some have supposed. Rather 
all the evidence points to the conclusion that pupils brought to bear 
all the mental abilities they possessed in their responses. Whatever 
information they gave, represented all they had to offer in meeting 
the requirements of the tests. 



84 



Stdiulnrdized Exercises in Ilis/ory 
HISTORY 







1 X 3 ^Y '^" ^7 a' ^ /<» // /A.f3'^ /^/Cf7/^iJJ^ 7- 




/ 


1 f 1 








• i ;^i/ 












X 






^- 












' 1 ' 
1 1 , 



















2; 












! : ! : 



















V 












! 1 1 1 
' > 1 1 



















.r 












j/i ! : 












1 






^ 












1 1 ' 

' ! ' ! 



















7 










/ 


^i 1 i i 


/ j 










H 






^ 












/;/!/ liM / 


/ i' 










t 




»-- 


f 








\% 


/ 


/<z\^\z.\ 


^1 


/ 








» 7 




«i 


/^ 








f/ 


3 


3\3\^ll\^ 


/! 










X 1 




fC 

L-''! 


// 








/| 


1 


^\^\^'J\J 


y; 




/ 






Xt 




^^ 


ik> 












X\^o\^\f:^ 












5 1 




t: 


13 










X 


^^'^W\Ci\^ 


2>\ 


/ 








d^ 






n 


! 1 ^ 






5 




/\Jb 










3 S 






if 








t 


\l 










r4 






fi. 








1 


^:^:^;3!;i 


1/ 














'I 








/i 


3 


-L^^i-i-l^liV 
















fi 












!^.3| ;/ 




/ 








7 






'1 












I ; ! :/ 












/ 






2n 












1 1 ' 1 




















X 3 '^^C^SSHOiil^t^ib 5" 3 / ;i^^ 





TABLE XV 
Showing correlation between the Pressy Rote Memory Test and History Test II 

Another point in this connection has been developed by Buck- 
ingham in an article entitled, "A Proposed Index of Efficiency in 
Teaching United States History" where he calls attention to the 
self-delusion of teachers and pupils in regard to the use of the so 
called higher processes in studying history. Instead of disco^'ering 
certain special relationships in historical knowledge, only informa- 
tion has been employed after all. The examination question may 
contain such expressions as, "What do you think," "Compare and 
contrast," "How do you know," etc. Yet the rating of pupils has 
been based largely on the iacts produced. 

V'an Wagenen has devised scales based upon information and 
character judgment. He finds a high degree of correlation existing 



Use for Supervisory Purposes 85 

between the results of these two kinds of tests. Buckingham con- 
tinued the study of this same relationship and goes so far as to 
establish an index of relationship between the two sets of tests 
whereby one can predict the ability in "Character Judgment" scores 
from the actual scores obtained through the "Information" tests. 
All the data obtained by this writer point to the validity of using 
carefully worked out informational tests as measures of historical 
ability. 

In the light of the successive points just given it is clear that 
the hasty criticism of history teachers mentioned at the beginning 
of this section is not valid. It is equally clear that the present 
tendency to get away from insisting that pupils have at ready com- 
mand a lot of facts and details is not sound. The writer has wit- 
nessed teachers insisting that pupils think in the history recitation 
when as a matter of fact clear thinking was impossible, for the 
pupils had no facts on which to base their reasoning. This should 
not be interpreted as a plea for many facts taught in isolation, but 
it is a plea for teaching many more facts than are now taught, in 
meaningful connections and to some purpose. 

In further support of the issue just raised an abstract is pre- 
sented of the aims in teaching history as formulated by the New 
Committee of Eight: 

1. Free the mind from the trammels of time and place. 

2. Produce open-mindedness. 

3. Induce patient inquiry before passing judgment. 

4. Give the methods of investigation and tests of accuracy. 

5. Develop that form of judgment which deals with the shift- 
ing and conditional relations of men in society. 

6. Yield high moral and ethical concepts. 

These aims deserve the support of every teacher of the subject. 
In the accomplishment of them more stress must be placed upon the 
weight of information than is done in practice according to the 
results of this study. A little consideration of this point impresses 
one with the absurdity of expecting pupils to think intelligently 
without a clear definite knowledge of the content of the subject. 

Sec. 3. — Significance of the Value Found for Each Exercise. 
Four Different Kinds of Historical Material. It is evident that the 
values assigned to any of the exercises can be modified by departing 



86 Standardized Exercises in History 

from the standardized conditions of the present. All that is claimed 
for these values is that they represent the relative difficulty, and 
specific difficulty of these points in Colonial History under the con- 
ditions in which they were obtained. In other words, so long as 
this period of our country's history is presented in schools as it is 
now, there being no appreciable difference in methods, types of 
teachers, textbooks, supplementary reading material, hygienic condi- 
tions ; and the results tested by the means used in this study, and 
confined to pupils of the eighth grade about to enter the high 
school, — these values are correct for large groups. This may appear 
to some readers as apologetic, but no more can be claimed for any 
standard test in principle. A class of exceptionally bright pupils ; 
a selected class which has unusual interest in historical studies ; or 
a teacher who is exceptionally strong in motivating the subject ; any of 
such conditions will modify the values considerably. But such con- 
ditions are not applicable to the general run of history classes as 
one finds them in the public schools of today. Only unselected 
pupils, unselected teachers and unselected material were considered in 
obtaining the values here presented. 

Because the responses came from eighth grade pupils about to 
enter the high school, and in most instances these same pupils had 
completed this part of history a year previous to the test, the writer 
is led to conclude that the work of these pupils represents approxi- 
mately the net tangible results of the history instruction relating 
to the facts usually taught in schools. If there is any inherent 
differentiation in types of material it ought to appear after the 
immediate advantages of drill and the factor of recency have been 
minimized. If this conclusion is valid the next point is of special 
interest. 

The weighted values of the exercises in the various tests indi- 
cate that there are approximately four types of historical material 
of an informational character now taught in the seventh grade 
relating to the Colonial Period. Table XVI was made up from the 
results of Tests I, II, III, and Lists X and Y, and the sigma values 
in Table XIV. 



Use for Supervisory Purposes 87 

TABLE XVI 
CLASSIFICATION OF EXERCISES ON BASIS OF CONTENT 

Political Values 

Mason-Dixon Line 2.46 

Grand Model 3.49 

Proprietary Government 3.03 

Charter Government 3.03 

First Representative Assembly 3.43 

Ownership of N. J. 3.39 

Philadelphia a Political Center 2.46 

First Written Constitution 3.49 

Town meeting of New England 2.76 

Treaty of Paris 1763 3.04 

Pitt in English Parliament 2.38 

Woman Suffrage in N. J. 3.05 

Lord Delaware Governor of Va. 3.12 

Dates 

Slavery introduced 2.26 

Landing of Pilgrims 2.37 

Settlement of Jamestown 2.05 

Period of Colonization 2.50 

United Colonies of New England 3.35 

Last colony founded (century wanted) 3.08 

Religious 

Decline' of Puritan Church 3.33 

Quakers founded Pa. 1.95 

Witchcraft 2.12 

Toleration Act of Md. 2.77 

Huguenots settled Charleston 2.77 

Puritan persecutions in Mass. 2.28 

First settlement of Catholics 2.34 

Social 

City of Brotherly Love 1.41 

English most numerous nationality 1.64 

Indians the first inhabitants .51 

Dutch settled in New York 1.37 

Harvard College 2.37 

Poor Debtors of Ga. 2.29 

Emigration Westward 1.94 

First houses of logs .73 

Pilgrims came from England .82 

Unexpected attacks of Indians .77 

Common means of travel 1.23 

All the colonists from Europe 1.08 

Pocahontas an Indian girl 1.22 

Candles gave light at night 1.97 

Colonists all under English control .97 



88 Standardized Exercises in History 

From the above it seems quite clear that teachers of history 
need to give more attention to skill in teaching political matters 
than is now devoted to them. Nine exercises referring to political 
events were found whose value is 3.00 sigma or above. It is the 
exception to find such content having a value of less than 2.50 
sigma. All the dates in the tests have a value 2.25 sigma ex- 
cept that referring to the settlement of Jamestown and that is 
above 2.0 sigma. Religious events are about as difTicult to learn 
thoroughly as dates. The m.edian value of social events given in 
the table is 1.23 sigma. This feature of the study points to the 
fact that teachers may profit by a careful analysis of the material 
of instruction, and then stress each type in proportion to its relative 
difficulty in learning for permanent retention. One may profit in a 
most practical way by a knowledge of the fact that different kinds 
of material show corresponding degrees of inherent difficulty for 
learning and teaching. 

The weighted values also indicate that teachers need to be more 
specific in what they expect pupils to learn, or need to teach pupils 
methods of inference in thinking. Just because pupils are taught 
that the Mayflower vessel started from Plymouth, England, and 
that the Atlantic ocean is between North America and Europe, is 
no guarantee that pupils will work correctly Exercise 4 which was 
included in List Y, "By the time the Mayflower had arrived at 
Plymouth Rock it had crossed the Atlantic, Antarctic, Arctic, Indian, 
Pacific ocean." Approximately 6% or 148 pupils failed on this point 
which seems so easy that every one would be successful. The pre- 
ceding exercise in the same list illustrates the same point. This 
reads, "Every one of the colonies was founded by men from Africa, 
Asia, Australia, Europe, South America." It is not likely that such 
a specific question as this is ever asked by teachers. The matter is 
just taken for granted that pupils know such facts from the other 
facts they have been taught. However, when enough pupils fail on 
such a simple point to make five good sized classes, it is evidence 
that all pupils can not be trusted to make inferences safely without 
training. 

To know the points just referred to above pertaining to the 
different types of material, and the need for specific training in 
inference is alone worth all the effort of this investigation. They 



Use for Supervisory Purposes 89 

are an outgrowth from the primary purpose of the study but none 
the less valuable. Standardized tests and scales are valuable in 
school administration, but the interpretation of them is the most 
important element. They need to be scrutinized in the light of all 
that is known about the principles of Education, its history, its 
administration and the psychology of learning. The results of their 
use is but the beginning of thinking; not the time to record scores 
and pass to something else. The scientific educator of the future 
will think of his instruction material much as the physician does of 
his medicines. He will not prescribe previous to an intelligent 
diagnosis. 

Sec. 4. — Possible Uses of the Standardized Exercises. These 
tests and scales are submitted to teachers and supervisors as con- 
venient standardized testing material for finding out how much 
information a pupil or a group of pupils has concerning the details 
of the Colonial Period of American History. If administered in 
standard form it can be ascertained in the time of a single class 
period or less, just what facts any pupil knows or does not know 
well enough to use immediately. There can be no question about 
the issue on the part of any pupil, parent or teacher about the re- 
sults, — they are all objective. 

They may also be used for rapid review work, after the usual 
instruction is given, as all the facts usually presented in an ele- 
mentary course are included in the measures. 

They may be used also for comparison of the results of instruc- 
tion by schools and by grades. It would seem that seventh grade 
pupils just completing this period, or eighth grade pupils who have 
just reviewed the work ought to make higher scores. In so far 
however as seventh grade pupils were tested they did not make 
higher scores. When tested side by side the eighth grade pupils 
made the better scores. It may be that the maturity and increased 
skill of the latter in reading, judging, and taking tests gave them 
an advantage which offset the disadvantage of the lapse of time 
since studying these facts. 

An equally useful purpose is served by these measures when 
they are used by teachers to diagnose the deficiencies in individual 
cases, or classes. The beginning of the information needed in 



90 Standardized Exercises in History 

diagnosis is afforded by a tabulation of the errors. This may be 
done in the time of a single class period by the assistance of pupils 
in some such form as here given. 

TABULATION OF WRONG RESPONSES MADE BY 150 PUPILS 
TAKING TEST I 



1. 



Mayflower 




7. Witchcraft 




chapel 




Baptists 


6 


hall 




Catholics 


34 


hotel 




Dutch 


20 


plant 




Indians 


19 


queen 


1 


8. Owners of New Jersej 




Standish 
Dutch 
Puritans 
Swedes 


28 

50 

3 


Conn. 

Del. 

Ga. 

Md. 
Mass. 


11 
30 
12 
15 
18 


Williams 




N. H. 


14 


Ga 


15 


Pa. 


10 


Md. 


17 


9. Native food crop 




Mass, 


28 


barley 


1 


N. C. 


1 


oats 


6 


N. J. 


6 


potatoes 


21 


Pa. 


7 


rice 


14 


S. C. 


3 


rye 


3 


Va. 


4 


wheat 


2% 


Patroons 




10. Mason-Dixon Lin« 




fishermen 


18 


Del. 


47 


fur traders 


58 


N.J. 


7 


teachers 





N. Y. 


2 


preachers 


16 


Ohio 


18 


Hooker 
Del. 
Ga. 
Md. 
N.J. 
N. Y. 


35 
16 
15 
11 

5 


11. Bacon's Rebellion 
Argall 
Andros 
Dale 
Minuit 
Winthrop 


3 

31 

2 

7 

28 




Yeardley 


3 


Harvard 




12. Charter Oak 




Brown 


S 


church 


6 


Penn 


7 


meeting house 


23 


Princeton 
W. and M. 


9 

38 


painting 
school 


3 



Yale 


16 


town hall 


10 



Use for Supervisory Purposes 



91 



13. Grand Model 






Md. 16 


Calvert 


22 




Mass. 15 


Charles 


35 




N. C. 5 


Smith 


57 




N. H. 20 
N. Y. 17 


14. Most numerous natic 


inality 






Dutch 


43 


21. 


Slavery introduced 


French 


6 




1607 21 


Germans 


4 




1620 29 


Irish 







1643 6 


Scotch 


3 




1683 7 


Swedes 


4 




1775 10 
1787 15 


15. Proprietary colony 








Mass. 


36 


22. 


New York founded 


N. Y. 


17 




English 27 


R. I. 


22 




French 5 


S. C. 


7 




Spanish + 


Va. 


23 




Swedes 


16. Professional class 




23. 


Tobacco in Virginia 


editors 


19 




corn 11 


lawyers 


30 




cotton 50 


physicians 


8 




indigo 7 


teachers 


9 




rice 3 


writers 


33 




wheat 4 


17. Toleration Act 












24. 


"Great Treaty" 


Episcopalians 


4 




Berkley 7 


Puritans 


46 




Endicott 


Quakers 


62 




Mather 


18. Oglethorpe in Ga. 






Smith 9 


Conn. 


36 




Standish 12 


Del. 


22 






Md. 


8 


25. 


Pocahontas saved Smith 


N.J. 


5 




Berkeley 3 


S. C. 


8 




Edwards 1 


Va. 


15 




Penn H 



19. 



20. 



Massasoit welcomed Puritans 
Canonicus 3 

Philip 17 

Powhatan 48 



Swedes in Delaware 




Conn. 


16 


Ga. 


10 



26. Charter government in 

Del. 33 

Ga. 7 

Md. 13 

Pa. 26 

Va. 38 



92 

27. 



Baltimore 

Boston 28 

New York 10 

Philadelphia 21 



Standardized Exercises 


in History 




Charleston 


Catholics 


19 


19 


Quakers 


5g 



28. Plymouth Rock in 




Conn. 


23 


Del. 


17 


Md. 


9 


N.J. 


5 


N. Y. 


1 


Pa, 


4 


R. I. 


20 


S. C. 


1 


29. Pilgrims came in 




1607 


51 


1619 


12 


1643 


6 


1683 


6 


1776 


2 


1789 


2 


30. Claiborne's Rebellion 


vs. 


Berkeley 


33 


Brook 


8 


Delaware 


12 


Say 


9 



31. Indentured servants in 

Del. 8 

Ga. 29 

Pa. 3 

S. C. 37 

32. Andros represented 

Continental Congress 37 

Proprietors 12 

Common people 41 



33. 


First Rep. Assembly 






1607 


14 




1620 


13 




1643 


24 




1754 


58 


34. 


Bradford leader of 






Baptists 


22 



35. 



36. 



37. 



38. 



39. 



40. 



Catholics' first settlement 

Conn. 14 

Del. 12 

Mass. ig 

N. C. 6 

N. H. 4 

N. J. 5 

R. I. 31 



Jamestown founded 




1619 




4 


1620 




24 


1643 




18 


1733 




21 


1754 




8 


First Newspaper 




New York 


Times 


28 


Philadelphia Ledger 


5 


Poor Richard's Al. 


83 


Providence 


Journal 


7 


Log houses 






brick 




5 


cement 







dirt 




6 


marble 




1 


Last colony 






Conn. 




8 


Del. 




5 


Md. 




4 


Mass. 




5 


N. C. 




6 


N. Y. 




6 


Pa. 




6 


S. C. 




22 


R. L 




17 


Va. 




5 


Emigration westward 




east 




29 


north 




12 


south 




14 



41. Last Dutch Governor 

Hudson 32 

Minuit 4 

Winthrop 17 

Yeardley 8 

42. Germans in Pennsylvania 

Md. 17 

N. J. 16 

N. Y. 39 

R. I. 9 

S. C. 7 

Va. 1 

43. City of Brotherly Love 

Berkeley 1 

Calvert 7 

Endicott 7 

Smith 6 

Standish 18 

44. Important Colonial War 

Queen Anne 16 

King George 16 

King William 16 

45. Government, form desired 

aristocracy 4 

autocracy 9 

monarchy 20 

oligarchy 4 



Use for Supervisory Purposes 

46. Quakers persecuted 



93 



47. 



48. 



49. 



50. 



Baptists 




24 


Methodists 




10 


Moravians 




13 


Reformed 




36 


Georgia invaded 


by 




Dutch 




45 


French 




34 


Irish 




2 


Swedes 




8 



Gorges control'd Maine 

Davenport 38 

A. Hutchinson 45 

Kieft 8 

"Holy Experiment" 

Wm. Bradford 17 

Wm. Brewster 15 

J. Leisler 2 

Minuit 23 

Natives of America 

Caucasians 2 

Chinese 

Malays 2 

Negroes 3 



These tabulations represent the actual responses of 150 pupils 
selected from 15 different schools taken at random. It reveals in 
an objective way how far short and indefinite is the aim to inform 
pupils accurately on some of the more commonly accepted points 
in the early history of our country. 

Such a table as the above may be easily made up by placing 
on the blackboard all the suggested answers to each exercise, and 
then call for a show of hands when any point is taken up. Suppose 
the exercise referring to the date of the landing of the Pilgrims 
were considered.* The teacher would ask, how many find 1607 as 



*After pupils had scored their own papers. 



94 Standardized Exercises in History 

the date selected; how many found 1776, and so on throughout the 
whole list. If a considerable number of pupils gave 1607 as the 
correct answer to this point it is evident that the class as a whole 
is not clear either as to the settlement at Jamestown or the landing 
of the Pilgrim Fathers. Both points need to be emphasized anew. 
But previously it should be the professional work of the teacher to 
find out why these dates were confused — the cause of the poor 
scholarship. Having first done this the remedial measures can 
usually be found. This is the use for which these measures are 
the most highly commended. They afford a direct, objective means 
of revealing to pupils and teachers alike the points of strength and 
weakness. They are shown so specifically that the means for over- 
coming the deficiencies seem almost self-evident. The check on the 
correctness of the tabulations was that the total of the reports should 
in each instance equal 150. 

Even with a selected group of superior pupils of the same abil- 
ity, classified according to the results of an intelligence test, these 
measures are an aid to the teacher. The sigma value accompanying 
each exercise indicates the inherent difficulty, and consequently it 
may be estimated at once whether much or little emphasis should 
be given. As was pointed out in the preceding section it seems per- 
fectly clear that there are some matters which need to be more 
skillfully taught than teachers have been heretofore aware of. 



CHAPTER VII 

SUMMARY AND CONCLUSION 

In the preceding chapters have been given the purpose and 
method of this study in detail, and a discussion of the results for 
educational purposes. In concluding the work, the whole study 
may be summed up and evaluated as follows: 

Sec. 1. — Summary 

1. The primary purpose of this study was the derivation of 
standardized measures for testing the historical ability of pupils in 
the elementary schools. 

2. Four such standardized measures were found possible: 
Scales C and D and Tests E and F. 

3. The content of these standardized measures is confined to 
informational material taken from the Colonial Period of United 
States History, 

4. All of the informational material usually included in an 
elementary course of study on the period considered has been stand- 
ardized. 

5. The content is incorporated in the form of exercises wherein 
the pupil must choose the correct one of several suggested plausible 
answers. 

6. The standardization of the exercises was based upon the 
performance of eighth grade pupils approximately one year after 
studying the Colonial Period. 

7. A method of procedure for standardizing testing material 
has been presented. 

8. Finally there is given a discussion of the use of the stand- 
ardized measures for supervisory purposes. The outstanding points 
in this chapter are: a reorganization of all the exercises into Test A 
and Test B, Part I and Part II with full directions for adminis- 
tering, scoring, using the results ; the value of the exercises ; four 
types of historical material ; and suggested means for using the exer- 
cises to discover clues for improving instruction. 

95 



96 Standardized Exercises in History 

Sec. 2. — Conclusions 

1. All the evidence of this study clearly shows that it is pos- 
sible to standardize all the testing material in the content subjects. 
The difficulty exists in devising a form for the material which admits 
of objective scoring. 

2. Many more scales and tests similar to the ones here pre- 
sented are needed in order that the results of instruction in each 
period or particular phase of United States History may be tested. 

3. The Standardized Exercises presented as a result of this 
study are reliable and will continue to be reliable measures until 
there is a decided change in the standard conditions under which the 
Colonial Period of United States History is taught. 

4. The results of this study indicate that abundant historical 
material is presented to pupils but little recognition is given to the 
ease or difficulty which pupils experience in learning the different 
kinds of material. 

5. The tendency to depreciate the value of learning thor- 
oughly much informational material in United States History can 
not be supported. Much well organized information is needed for 
constructive thinking if the new aims of teaching history are to be 
realized. 

6. The scales and tests here presented may be used to deter- 
mine both the achievement of pupils, and the remedial measures 
needed to improve instruction. 



BIBLIOGRAPHY 

Bac/lcy and Ru(fg The Content of American History as Taught 

in the Seventh and Eighth Grades. 

Ruck'tJHjIiam, B. R. Spelling Ability, Its Measurements and Distri- 

bution, 
A Proposed Index of Efficiency in Teaching 
U. S. History, Journal of Educational Re- 
search, Vol. I, No. 3. 

C.rdlf, A. L. Tables de Calcul, V. S. (leological Survev. 

Dr-zi-ry, John Democracy and Education. 

Kflly, F. ./. Teachers' Marks, Their Variabiliu and Stand- 

ardization. 

McCall, JF. A. A New Kind of School Examination, Journal of 

Educational Research, Vol. I, No. 1. 

Monroe, JF. S. Measuring the Results of Teaching. 

Riujg, //. 0. Statistical Methods Applied to Education. 

Starch, Daniel Educational Measurements. 

Fan JFaocncn, M. J. Historical Information and Judgment in Pupils 

of Elementary Schools, New York, Teachers 
College, Columbia University, 1919. 



97 



/ LIBRARY OF CONGRESS 



022 158 769 8 



LIBRftRV OF CONGRESS 



022 158 759 8 



